Keynote: the State of Drupal (founder Dries Buytaert)

session information: Keynote: the State of Drupal

Short version: Drupal Rocks!

a brief history of Drupal

• 10 years ago, Dries was involved with the Linux WLAN project, developed what was then called Dorp as a way to coordinate work on the network
• moved project to drop.org (unintentional bilingual pun), renamed Drupal
  
• people made lots of suggestions, so he made the project open-source
• Drupal 1.0.0 released January 15, 2001
• major geek sites such as kerneltrap.org started abandoning PHPNuke for Drupal in 2002
• first DrupalCon in Antwerp in 2005 with just 25 or so people
• legitimization of Drupal in business has invalidated Dries's theory on the growth of average facial hair among Drupalers
• first Drupal book by Robert Douglas has improved Drupal's legitimacy
• code & documentation sprints started in 2007 to improve drupal.org site
• redesigned http://drupal.org site will debut by next week, features AJAX search, drag-&-drop dashboard, custom tabs
  
• http://drupal.com has been just a landing page for years; Development Seed has created a sales site
• trademark costs are getting expensive; hope to cover costs with ads
• The Big Server Meltdown of 2005
  • needed new server, put up a PayPal button and raised $10K in 48 hours.
  • Open source institutions, Sun offered free server and services
  • booth at OScon Portland led to the founding of the Drupal Association
• Drupal 4 had drastic usability improvements
• Drupal 5 got auto installer
  
• DrupalCon Brussels 2007 had 150 people; Sunnyvale 2007 had 300, DrupalCon Boston had 900
  
• today there are 1400 at DrupalCon plus waiting lists and branch locations for those who couldn't get in
• each year has seen a doubling of downloads, contributed projects, and unique hits to drupal.org
• [examples of Drupal in pop culture -- many slides]
• "Built by everyone, controlled by no one, and it actually magically works: Drupal is like the Internet"
• "replace planning with coordination" -- like meeting up with people after vs. before cell phones

So where are we now, and what will be the big improvements in Drupal 7?

• Drupal 7 development opened February 2008, when Drupal 6 was released
• code freeze will happen September 1, 2009, 7.0 will be released "when it's ready"
• When will it be ready?  When critical issues count reaches 0
• Usability is most important change for D7
  • two installs provided: minimal and default with most common modules and dummy data
  • Usability Team has been doing formal testing since last year
• better file API allows better media & document handling
• Fields API replaces CCK and Profile modules - migration paths have not yet been worked out
  • raises question: should avatars, taxonomy, polls be applied to users as well as nodes?
  • field storage model is still being worked on
• code testing is another priority... roughly 10,000 tests already written, covering more than 78% of code, and all currently pass!
  
• database abstraction layer is retooled to support newer functionality, SQLite, and tests
• important to imagine the future of the Web
  • start thinking of the Internet as one big machine: web apps, cloud computing
    
  • steps of development: linking machines, linking pages, linking data, linking things in the real world

Drupal is a movement, so what is its mission?

• short-term: usability, great programming platform
• long-term: help bootstrap the future of the Web
• There is a difference between telling people what to do and creating a movement.
• A movement needs a healthy ecosystem, many leaders and contributors (not just lurkers)

Knight Drupal Initiative grant winners

• Knight Foundation asked for help last year identifying best Drupal projects to lower barriers to entry, provide powerful tools for digital publication
• winners will receive a total of $485,380
• Addison Berry will provide clear instrutions, run doc sprints
• Bill Fitzgerald will create free publishing system
• Dave Cohen wil allow anyone to create a one-click online news site integrated into Facebook for distribution
• Nick Vidal will develop personalized news distribution system
• Rob Loach  will create Drupal module for micro-blogging and cross-site aggregators for micro-blogging
• Development Seed will create mapping tools for geotagging and news aggregator

Keynote: Is Drupal Moral? (David Weinberger)

session details: Is Drupal Moral?

David Weinburger, co-author of the Cluetrain Manifesto and Everything Is Miscellaneous: The Power of the New Digital Disorder

• Why even ask? Because Drupal is used in high-visibility places (Howard Dean, Recovery.gov)
• Can technology be moral instead of amoral?
  • can only moral agents be moral?
  • do some things inherently suck, like nuclear bombs?
  • context can help us assign morality to amoral things
• hyperlinks are the new punctuation, telling us how to continue reading instead of when to stop reading
  
  • in the past, knowledge has conveniently, seemingly naturally divided itself into book-sized units
  • Britannica has 180,000 words on "philosophy," Wikipedia has 9,000 (longer than guidelines suggest it should be)
  • but Wikipedia article has links to related articles -- if you count them Wikipedia has much more
  • Wikipedia's shape is closer to the shape of truth than Britannica's
  • traditional databases organize information by reducing information, cramming selected bits in boxes and leaving out everything else
    
  • by linking out (to other sites), we allow each entry to take its own shape while still being organized
• moral principles vs. consequences
  • principles are top-down: make the current situation fit a pre-defined criterion
  • but even if people agree on the principle, they may not agree on how the situation fits! (ex: murder is wrong, but is abortion?)
  • utilitarian approach: judge the unique situation by whether its consequences are more good or bad
  • but anyone can think of a hypothetical case where any situation has good or bad consequences
  • moral intuitionism: trust our guts
  • but some people have really rotten guts
  • for the most part, people are sympathetic creatures and make moral decisions intuitively
  • when sympathy turns toward the world it becomes care
• back to Drupal
  • as a technology, Drupal lets us connect as people and care about each other
  • Drupal, the Web, and morality share a single architecture
  • but it doesn't happen by itself
  • contrasting view: Web allows similar people to cluster together and close their minds
  • but is the traditional newspaper any more diverse than Web sites? are most conversations?
  • small talk allows us to find a common topic where we mostly agree; then we converse about that
    
  • surprisingly, this fundamental question (does the Web make us more or less diverse) doesn't matter
  • we all learn from the Internet that the world is way more interesting and diverse than we thought
• The exciting conclusion
  • yes, Drupal is moral!  But that doesn't tell us enough.
  • regardless of the architecture, we must perpetually fight against our baser nature
    

Keynote: Our Online Identity (Chris Messina)

session details: Keynote: Our Online Identity

incidentally: Chris Messina founded BarCamp.

• Why are we still using the awful term "Web 2.0"?
• five rules:
  • the perpetual beta - a process for engaging customers
  • share and share alike
  • ignore the distinction between client and server
  • open APIs and standard protocols always win
  • "lock-in" comes from proprietary data and formats
• question is not, "When does Web 3.0 come," it is "does open source still matter?"  Not like it used to.
• technology is becoming humanized and humans are becoming technologized or "cyborganic" (example: iPhone)
• identity: who are you?
  • logins have little to do with who you are, just provide secret information
  • Facebook is assertive in suggesting people you may know
  • Facebook is also adamant about using real names instead of aliases, and for the most part people are OK with that
• The Social Web
  • "a social revolution in how people connect and share"
  • BarCamp and Coworking represent a desire of networked people to connect in person
• activity streams
  • Twitter meets a need Blogger does not because you're joining an existing conversation instead of trying to get people to listen to you
  • Dunbar's Number: the number of monkeys in a group who can keep up social ties
  • "...we're addicted to our friends.  When the computer lets us access our friends, we look like we're addicted to the computer" (Danah Boyd)
  • how do you follow this content?  RSS and Atom are not adequate
• anatomy of an activity: who did what to what where?
  • basic online activities to track: blog entries, notes, photos, videos, bookmarks
• The OpenSocial Stack: decentralized framework for sharing personal data among sites
  
  • Facebook stack is simple: identity, friends, feed -- APIs to tie into each of the three
    
  • recent proposal: identity, discovery, authorization, profile, friends, streams
  • Facebook lets you control what info is public outside and inside your circle of friends
  • assertion: the more control you give people, the more they will trust & use your service
  • more than 90% of Twitter accounts have been voluntarily made public by their owners
    
• performative identity
  • problem Facebook is having: shared content cannot easily be unshared or deleted -- sharing gives up control
  • Flickr problem: merging with Yahoo led to merging accounts (for legal terms-of-service reasons), which freaked people out
• Copyright vs. Creative Commons for online info
• Cloud computing
  • OpenID has the most potential in the cloud, where the desktop computer is not consistent
• Government
  • new movement toward transparency motivated by failure of opacity
  • recovery.gov is the most visible sign of open-source acceptance
  • open source as a model for government -- participation and agency
  • trust+transparency make open source work
• Work on stuff that matters!
  • work on stuff that matters more to you than money
  • create more value than you capture
  • take the long view
    

Drupal in the Newsroom

session details: Drupal in the Newsroom

Panelist introductions

• New York Observer has been running Drupal since 2007, recently moved to D6
• McClatchy Interactive (online editions of newspapers such as Miami Herald) has integrated as many as 50 different programs into Drupal
• Mother Jones Magazine decided to move to Drupal just a year ago, went live 2.5 weeks ago
  

Workflow

• important to shift workflow so that Web comes first, print comes later; use revisions, unpublished status to collaboratively edit drafts
  
• editors assign weights to stories to control order on the page
• Miami.com has expanded McClatchy's conception of how big & complex a Drupal site can be
• MoJo's primary challenge had to do with changing the culture & identity of the magazine to 24/7 blogging forum
• "reverse publishing" -- scrape RSS feed from Drupal into editing software
• writers have to adjust to writing more frequent, shorter pieces instead of long ones

User roles and permissions

• jobs are not always as well defined as Drupal would like ... some bloggers can be trusted to post directly while some senior contributors need editing
• start with reporters, editors, editorial producers, tech team
• roles are constantly evolving
• MoJo tends to err on the side of giving more permissions than people may need
• to allow writers to post unpublished content, set default in content type to unpublished

Engaging the public

• readers now expect to be included in conversation - commenting is crucial
• flags on comments at MoJo: recommended solution, documentation of a result
• have to create culture of commenting to overcome barriers to entry (registration)
• MoJo found comment volume went down 90% when required registration
• Mollom and Flag modules can dramatically reduce comment spam
• encourage authors to respond to comment thread

Subscription management

• no good solutions to offer; all sub mgrs seem to interface badly with the Web as yet

Why Drupal?

• the community (of developers)
• open source values
• at McClatchy, publications started demanding it once they saw it in action on other sites
• quick development time -- week and a half from new concept to public site
  
• bang for the buck
• question: is this a fad?  Will we all be switching CMSes in 2 years?  Community is looking forward to the future of the Web more than most.

Newspaper group at http://groups.drupal.org

Building Church Websites with Drupal

session details: Building Church Websites with Drupal

• Some churches use CiviCRM, but it has some security issues
• when choosing between CiviCRM and a church management system, go with the church mgmt system
• possibility: sync Drupal/Ubercart with Fellowship1 as a backend
• http://Geeksandgod.com is a good, easy place to learn Drupal
• churches are not businesses! but many church sites look like business sites.
• churches are not nonprofits either, they are communities, not organizations
  
  • share life, not information
  • public-facing side should be a small part of the site
  • emphasize user-submitted content and interactive media
• how not to do it: list the services provided offline by the church
• how to do it: Organic Groups -- anywhere people organize physically, they can organize online
  • instead of creating <h2>s, create groups
  • even if you don't use all the features right away, OG gives you a framework to do it later
  • instantly have newsletters
  • much easier to manage!
• setup
  
  • create a "ministry" (committee) content type
  • add fields and CCK stuff -- use email field to create a contact form!
  • designate as group node
  • use taxonomy to designate type, so you can sort them
    
  • theme a teaser for each type
• multimedia sermons
  • necessary modules: CCK, filefield, Views, FFPC (FileField PodCast), SWFtools (player), GetID3
  • create content type for sermons
  • create separate file fields for MP3, MP4, and flash video
  • configure SWFtools for each media type
  • insert SWFtools codes into desired places in node.tpl.php templates
  • create a View of sermons, and add a separate feed display for each media type
  • create a sermon_series content type and add node reference to sermon type
  • use views_embed_view to embed sermon list view in sermon_series page template
• session notes: http://geeksandgod.com/files/drupalcondc.zip
  

Business Analytics in Drupal with Views

session info: Business Analytics in Drupal with Views

Slideshow with presenter's notes: http://is.gd/lXYU
(Be sure to turn on the "Notes on Slide" tab, since the actual slides have almost no text).

• Recurring problems are fun: if you can solve 80% of them with a single solution, write a module!
• Views is a great query builder, but it doesn't do graphs
• submitted a request, but people kept wondering "is it even possible?"  Are PHP/Drupal "serious" enough tools for business analytics?
  • PHP sites: Flickr, Wikipedia, Wordpress, Facebook, Digg, Del.icio.us
  • Drupal sites: FastCompany, The Onion, NY Observer, UN WFP, Recovery.gov
  • "Java is more serious: they have longer beards!"
• Views_charts module just contributed this morning, 6:00 AM!
  
  • just a display plug-in for Views
  • front-end for OpenFlashCharts (Flash), will soon also implement AmCharts and GoogleCharts (not Flash)
  • you can also write your own charts
• except: it's not all that easy (yet) due to SQL grouping and aggregation.
  • need views_groupby module, contributed yesterday!
• Panels, CustomPage, and Context help lay out pages
• What's next?  No reason why Views should only work with SQL... why not SPARQL?

The Business of Open Source

session details: The Business of Open Source

• why should business care about open source?
  
  • because free software can be turned into actual jobs: Acquia, Lullabot, et al.
  • because it's a highly successful model that can be applied to the way you do business
  • because it's growing at a time when the economy is failing
    
• free redistribution -- good ideas should be shared
  
• source code -- anyone should be able to review the guiding principles behind an idea
  
• derived works -- any idea needs to be able to grow and change and die
• integrity of the author's source code -- credit = responsibility
• no discrimination against people or groups -- diversity of experience and input is tremendously valuable, so evaluate the idea, not the person
• no discrimination against fields of endeavor -- it's OK to be from marketing (or wherever)
• distribution of license -- cut out the red tape! simple is better
  
• license must not be specific to the product -- if an idea is good, implement it across the board
• license must not restrict other software -- let ideas grow independently of each other
• license must be technology-neutral -- don't tie the idea to a specific implementation
  
• on the other hand, open source needs to learn it's OK to earn a profit!
  • no one at Lullabot has any formal business training
  • do some research, find out what others charge for similar services
• B Corporations: a proposed corporate type whose directive is to be beneficial rather than profitable
  • movement has stalled and has no legal standing
• ways to start
  • pick a priority
  • identify the benefits
  • identify the challenges
  • don't wait!
• Lullabot books its employees only half time, considering the other half of their time given back to the open-source community

Providing Drupal Training

session details: Providing Drupal Training

presentation materials, discussion, etc. will be posted at the session info page above.

Lee Hunter's notes from his portion: http://streamoflight.com/content/training-and-documentation

http://Drupaltherapy.com provides Drupal-specific training for beginners, admins, specific needs (knowledge gaps due to previous learning)

• new Drupal learners face different issues than current users, depending on their background and what they intend to do
• advanced training for when you hand on a project to its new admins
• case study: Chicago area police dept replacing their radio dispatching system with Drupal and laptops in cars
• assess knowledge and needs of your clients
  • what do they already know?
  • what do they need to know?
  • how much do they need to un-learn?
  • are there different learners in the same pool?
• find entry points for learners
  • learners will progress at different rates and in different directions
• create meaningful curriculum and resources
  • cover concepts in best logical order
  • complete understanding of A before moving on to B
  • expose concepts and methods before punching buttons
  • collect feedback on the progress of the learner
  • prepare learners to learn more!  teach them to navigate learning resources; offer mentoring or tutoring
• showcase of successful training models (best practices)
  
  • support group model -- get people in a room, group similar problems together and address together
  • post anything that comes out of a training session on your Web site
  • focus on how the learner defines success, don't just iterate through the user interface

Drupal and the Geospacial Web

session detais: Drupal and the Geospacial Web

• Google Maps is a good start, but look is not customizable
• custom modules can render SVGs but require a lot of work and maintenance
• Mapnik http://mapnik.org is open-source platform for producing "beautiful maps"
• 5 problems to resolve:
  • language -- interface of Google Maps is translated, but map content is in English and sometimes other languages intermixed
  • history -- Google Maps only shows today's borders
  • usage -- Google Maps assumes user should be able to explore the map and wants to see streets, etc.
  • privacy -- information is not always public, but Google Maps cannot be behind password-protected sites
    
  • look -- Google Map may not meet the client's expectations or informational needs
• example: Stumble Safely
• Mapstraction or Nice Map modules plug into Google but also other mapping services
  
  • Cloudmade (beta) can customize look of maps from Open Street Maps data
  • FreeEarth display for satellite imagery
  • NASA display
  • WMS = Open Geospatial Consortium Web Map Service -- opengeospatial.org
  • http://openstreetmap.org
  • http://openaerialmap.org
• Drupal should be negotiable too -- may need to map data from other sources
• WMS is great, but request & response format is very complex
• KML is a simple format, now stable; also GeoRSS
• AtomPub-Geo allows geo data to be subscribed to by other sites
• how to query?  OpenSearch standard for structuring queries; OpenSearch-Geo for geo data
• where to get data?  Flickr, Panoramio, Yahoo! Search, Wikipedia, Twitter, local governments, global NGOs, http://geonetwork.org
• repositories and aggregators: http://geocommons.org
• another reason not to use Google Maps: data ownership -- they own any data you map!
• http://opendatacommons.org - open database license (currently in development)
• where do these data come from?
  • O'Reilly "Geowankers" team -- so named so they won't appear in any major media
  • http://osgeo.org - Open Source Geospacial Foundation
  • data mapping parties and geocoders
• A Drupal example - what might you do?
  
  • real estate data from Trulia, bar locations from geocommons, weather from NOAA, locations from GPS enabled devices
  • exchange info with other CMSes via GeoRSS

Making Dates and Calendars Rock!

session details: Making Dates and Calendars Rock!

Karen Stevenson, maintainer of Date and Calendar modules

using demo module to demonstrate, organic groups

• Date Tools admin screen (under Content) allows you to set up a bunch of features at once that used to be scattered
• creates a content type, field, and View (with page & block displays) automatically!
• default values for From & To dates can be specified relative to current time or each other
• site-wide custom formats for date display
• default display of calendar is determined by granularity setting
• year range is important to keep bots from going through your calendar forever -- can be absolute or relative
• multiple date fields can be shown on the same calendar, using colors to distinguish
• tons of localization improvements
  

Practical Semantic Web and Why You Should Care

session details: Practical Semantic Web and Why You Should Care

video by Robert Nelson Vance

• The Semantic Web is for propellor-heads with Ph.D.s
• it's really about bringing more meaning to the Web.
• RDF = Resource Description Framework = "robot food" -- turn human-readable information into machine-readable, distinct XML format
  
• RDFa is the standard adopted October 14, 2008 by the W3C allowing RDF to be embedded within XHTML pages
  • both formats are searchable with SPARQL, but RDF is usually more appropriate for large quantities of data, to strip out the non-semantic text
    
• sometimes Apple is a fruit, sometimes a company, etc.  links need "flavor"
  • in HTML, can use <a rel=> to give links flavor
  • RDFa allows you to use <a property=>
• how to add these semantic tags to your pages?
  • OpenCalais is a top-down process -- point it at your site, and it adds tags
  • bottom-up approach is to add tags as we create themes, pages, etc.
  • Zemanta module will automatically suggest relevant images for pages as you write them, and tag them appropriately
    
• http://Recovery.gov is using Drupal 5, does not currently support XML output, but that is planned
• StumbleSafely, dc Bikes use open data and Drupal to create new information
• DC's Digital Public Square is "pure robot food" -- needs processing to make it palatable to humans
• in the past, we moved data to where the processing is (desktop computers); in the future we will move processing to where data is (web servers)
  
  • Internet will be like the public library system, but for robots
• Drupal is on track to be the first general-purpose CMS with significant semantic data out of the box.  D7 will be the most widely deployed semantic platform as soon as it is released.
  
  • existing sites (dbpedia, FOAF profiles, etc.) have over 2 billion RDF triplets.  D7 will conservatively produce that many each month, probably more.
• Views in the future could (will) use SPARQL to retrieve data from other sites
• maybe you care about money: "blog" stands for Better Listings on Google.  Post to a blog, your ratings go up
• existing modules: RDF, RDFCCK, SPARQL, EVoc, Exhibit, Exhibit Views (?)
  
• MOAT module currently only for D5, allows free tagging to be semantic
• go forth and experiment today!
• Dries said every movement needs a mission -- Drupal's is to help evolve the Web.
• question: how can we trust the information people have marked up?  Won't spammers exploit RDF?  Distributed identity & trust networks to the rescue... but there will undoubtedly be more problems to solve.
• http://groups.drupal.org/semantic-web
  

Ubercart on Drupal 6

session details: Ubercart on Drupal 6

• goal: out-of-the-box e-commerce module bundle
• Commerce Guys: new Ubercart consulting company
• MST3K http://rifftrax.com site uses Ubercart and was the reason for file download capability -- use coupon code DRUPALCON for 20% off!
• all work so far is just laying a foundation -- it works, but needs optimization
• what's new in D6:
  • performance enhancements
  • usability improvements
    
  • reduced module dependencies
  • more fine-grained permissions
  • code cleanup and bug fixes
  • core API improvements -- didn't know the right way to do things at first
  • support for conditional actions -- replaces dependency on Workflow-NG module
  • more contributors to core!
• what's holding up 2.0?
  • conditional actions support
  • user feedback -- what doesn't work?
  • update functions -- haven't even been written yet, so don't update a live site!  Data should be safe, but workflow and other functions will not update.
  • core API support for international taxes - need feedback
  • issue tracker cleanup -- evaluate outstanding issues
  • it is working, people are using it
  • come to the code sprint and help!
• optimization, internationalization, innovation!
• simplify the selling process: put buttons on nodes and user roles to make things and roles available for sale
• lead the way in "semantic commerce" -- make catalog items machine-readable by default, without need for extra modules
• Ubercart is not a stand-alone e-commerce application -- it is intended to be part of Drupal
  
• not the perfect e-commerce solution, probably won't be
• not an e-commerce company
• it is "community commerce" and we're all part of it!
  

Advanced Ubercart Usage

session details: Advanced Ubercart Usage

demo site using Sharp Sales theme

• use a single product node to sell multiple items, example: DVD, download, or both
  • delivery attribute with options and features assigned to those options; download option not shippable
• selling node access with uc_node_access module as opposed to role (so you don't have to have a role for each piece of content)
  
  • create node type for premium content, use content_access to restrict access to role
    
  • set time delay, time period, expiration date
  • use uc_restrict_qty to limit buyers to one of these items
• registration forms -- instead of using text attributes, use uc_node_checkout to make it a node type so you can use Views to list registrations
  • create a registration node type, add fields with CCK, including status and order fields to hold order info (as text)
• conditional access -- develop using the admin, but best not to keep PHP code there
  

Mystery Showcase Site - Higher Education / Museum

session details: Mystery Showcase Site - Higher Education / Museum

Mystery site unveiled: Sam Fox School of Design and Visual Arts at WUSTL -- currently at http://beta3.palantirbeta.com

• school actually has five branches
• design by Studio Blue presented challenges
  
  • two-pane design with sliding effect for simultaneous presentation of information and "inspirational content"
  • looks awesome on an iPhone!
    
  • multiple instances of Zen layout inside each other to display two nodes in a single page
  • all good designers "cheat the grid" -- Zen gives you that flexibility
  • two-node layout was designed to degrade gracefully in case client decides to abandon side-by-side layout
  • left-hand pane is cached so that you can change the right pane without reloading the left (and without frames)
    
  • no core was hacked in the making of this layout!
  • non-web-safe fonts are images substituted in with jQuery, so they degrade to plain text
    
• new modules to be released very soon
  
  • Flashy: a GPL multimedia player for locally hosted video!
    • player module for .flv files with Filefield formatter
    • still somewhat basic
  • SWF Embed: because SWFObject doesn't get along with jQuery
  • Views Date Range
    • allows arbitrary date ranges for views
    • was incredibly hard to accomplish -- very ugly SQL!
  • Views Cycle
    • makes slideshows out of lists output from views
• what's next
  • site will launch later this month, modules released next week
  • Mildred Lake Kemper Art Museum site will be a twin site on the same Drupal install with Domain Access

Documentation is Hot!

session details: Documentation is Hot!

video by Chris Rowe

If anyone has suggestions for how to make documentation sound actually sexy, they are welcome!

• What are docs? 
  
  • Drupal Handbook
  • API site pulls from "doc blocks" of comment in code
  • help screens provided by modules (also in code)
    
• Being a team
  • everybody with a drupal.org account can now edit the Handbook!  (revisions are enabled)
    
  • credit for editing will now be chronological instead of alphabetical
  • there are still some elevated permissions, for rolling back revisions, etc.
• Accomplishments
  • handbook has gotten much better
  • reorganized the theme guide for D6
    
• The Road Ahead
  • new help system with useful help stored in HTML files in a folder called help
  • downloadable Getting Started Guide, etc. in modules
  • utilize tagging and views to provide multiple ways of organizing the docs, instead of just Book module
  • ability to flag forum posts/comments for inclusion in the docs
    
  • CVS versioning for online docs so sites can be synced
  • Goal: D7 should ship with full documentation, instead of 6 months after like D6
  • teach people that making patches is not so hard
  • internationalization
  • redesign of drupal.org includes new documentation landing page
  • lots of sprints! all the cool kids are doing it! combine with your DrupalCamps! funded by Knight Foundation!
    
  • newbies are encouraged to help!  Read the docs and make notes!  See if the how-tos work!
  • how about some screenshots and videos in the docs?  Well, currently you have to be a doc admin to upload files...
  • Let's make it less of a pain in the ass!  We have 4400 pages of docs!
• Getting to work
  • click edit and make a change! it's much easier to edit info that's there than to find info you haven't got!
  • look in the issue queue for ideas
    
  • come to the doc sprint on Saturday!
    

Search Engine Optimization

session details: Search Engine Optimization - slides from the presentation (PDF)

SEO tactics:

• research: understand the target users... imagine individuals and brainstorm keywords
• site architecture: navigation should be text, not images, and include keywords (not click here)
• semantic markup:
  • use tags that are meaningful to bots, alt tags, etc.
  • explain importance to clients so they can maintain it
  • remove non-semantic markup from GUI editors
• content strategy: search engines want to showcase sites with quality content, so make sure your pages deliver as advertised
• inbound links: most time-consuming & difficult part of process, probably 80% of effort
  
• register domain name for 10 years in advance: search engines look at expiration date as a sign of legitimacy
• important Drupal modules:
  
  • SEO checklist -- provides a list of things to improve
    
  • taxonomy -- important because it puts keywords and reciprocal links on each page
    
  • page title -- make sure it's meaningful!  Most important tag on the page for placement.
    
  • Global Redirect -- prevent same content appearing at multiple URLs (also be sure to do this with multiple domain names)
    
  • pathauto, path redirect -- reduce or prevent 404s
    
  • meta tags -- actually not very important for placement, but can improve click-through rate
  • xml sitemap (still in development for D6)
    
  • google analytics
• Google Webmaster Tools highly recommended, and Google SEO Handbook (PDF)
  
• Selling SEO -- what are clients missing? emphasize long-term rewards
• what do you sell?
  • research
  • keyword analysis (again, Google Webmaster Tools)
  • content strategy
  • training
  • measurement and evaluation
• resources:
  • http://groups.drupal.org/search-engine-optimization
    
  • http://civicactions.com
  • http://volacci.com/drupal
  • http://GrowVS.com
  • http://SEOmoz.com

Frontend and Backend Optimization

session details: Frontend and Backend Optimization

Front End Optimization

• lost most of my notes!  :-( :-( but slides will be online.
  
• why measure? so you can prove your progress
  
• what to measure?
  • most useful metric is when the page becomes usable -- when the DOM is loaded.
    
• how to measure?
  • Firebug's Net tab
    
  • YSlow plugin for Firebug
    
  • AOL Pagetest
  • WebKit.org
  • all of these tools will reveal that load time is a small factor compared to request & response time.
    
• what to do?
  • reduce requests! each file is an HTTP request.
    • use sprites to reduce background images
    • aggregate scripts and styles using Drupal's performance settings
    • no redirects! at least not for supporting files
    • caching
    • use CSS border-radius instead of images for rounded corners
  • use a CDN (content delivery network) if possible
    • comparatively cheap, starting at $0.07/GB
    • reduces round-trip time to server
  • controlling browser caching
    • with Apache default settings, browsers check each piece of content on each load.
    • set expires headers to far future to prevent this
  • GZip content (text, not images) often more than 50%
  • move all CSS to the <head> for faster loading
    • loading CSS later will cause page to flash as it is re-rendered
  • move scripts to the end, right before </body>, because loading scripts blocks page rendering.
    • for this to work, you can't use any onFoo handlers in page (onClick, et. al.)
  • minify CSS and JS
    • remove comments and whitespace
  • Parallelization
    • use multiple host names so browser will request content in parallel
      
    • browsers usually do not have more than 2 connections open to a host at a time
      
  • reduce image weight -- hidden content such as comments in image files -- http://smushit.com and Firefox add-on
  • move static content to a cookieless host so that cookies need not be sent with each request

Back End Performance Optimization for Large Sites

• slides will be posted to the session's page after
• "performance" -- faster for individual pages? handle traffic spikes?
• define objectives first
• there will always be diminishing returns, so start with low-hanging fruits
• proper diagnosis is essential
• validate the results on a test server
• multiple cores on server are helpful - the more the better
• throwing RAM at the problem can be very helpful, but only if that's a limiting factor
• same with multiple servers -- setup and maintenance are not cheap.
• tuning a system can avoid or delay a server split, saving a ton of money.
  • avoid OS bloat, install only what you need -- if you're using MySQL, remove PostgreSQL
    
  • avoid Apache bloat too -- remove unneeded modules such as Python
    
    • be sure to have mod_gzip running
  • lighter-weight alternatives to Apache: lighttpd, nginx
  • consider pros & cons of MyISAM vs InnoDB for each table -- use InnoDB for sessions, watchdog
    
  • Query cache is most important thing to tune! also table cache, key buffer
• MySQL replication is effective but difficult until Drupal 7.
• always use a recent version of PHP!
• install an op-code cache / accelerator such as Zend (commercial) or APC (stable) or eAccelerator (fast)
• run PHP as mod_php or FastCGI, not CGI
• drupal-specific tips:
  • mainly db intensive, with 100s of queries per page, so tune MySQL first
    
  • make sure cron runs regularly
  • use throttle if you can, but be wary of its effect on cache
  • try not to call things over the network in your modules! cache data, use job_queue and queue_mail
  • caching, but helps anonymous users only.  Boost helps a lot, but make sure you truncate sessions after install
  • block caching in D6
  • memcached helps a lot, but requires a lot of setup
  • slow modules: statistics, gsitemap, tagadelic when used with lots of terms, node_access with large numbers of nodes
  • to monitor problems: top/htop, vmstat, apachetop, slow query log
  • stress testing -- for test servers only!
  • 
    

Drupal 7

session info: Drupal 7

Webchick got started with Drupal at 2005 Google Summer of Code, now is Drupal 7 Maintainer

What happens during the release cycle?

1. release the current version
2. open next version for development ("head") and retire old version (5 will retire when 7 released)
3. code thaw -- fix things that have always annoyed you!  add features!
4. code freeze -- slated for September -- just kill bugs, make it stable
5. return to step 1
6. port all modules :-(

Should I start building production sites on D7?  No!

Should I start porting modules to D7? Yes!

• use Coder module to help you port modules!! Now available for D7!

Drupal is a "do-ocracy" -- what gets done is what people do

Core Maintainers: Dries, Neil Drumm (D5), Gabor Hojtsy (D6), Angela Byron (D7) -- but they just supervise -- others do all the work and get little credit

What's New?

• Changes for site builders
  • new minimum requirements: PHP 5.2, MySQL 5.0, PostgreSQL 8.3
  • installation profiles (minimal / default)
  • SQLite support
  • new password strength indicator
  • time zone handling in core
  • story content type renamed article
  • default taxonomy tags
  • add-a-menu link
  • permission screen easier to read, each has descriptions that can be translated, grouped by content type
  • users can cancel own accounts
  • improved translation interface
  • forums have new pretty icons
  • content types now under site building, some new options available
  • modules page contains links to help pages
  • Ping and Throttle modules removed; user-customizable comment settings removed
  • configurable error reporting with easy to understand options
  • Advanced Help moved into core
  • usability testing by Mark Boulton Design, U of Baltimore
• Changes for designers
  • page.tpl.php redesigned with wrapper classes, better CSS development without hacking PHP files, based on Zen theme
  • new theme: Stark -- like Chameleon, but with even less defined
  • improved javascript functions
  • cross-library compatibility (for jQuery) -- already works in D6
• Changes for coders
  • SimpleTest module greatly improved (also in D6)
    
    • writing tests causes you to think problems through, find obscure bugs
    • know instantly if you've broken something
    • fix a test along with your bug fix, and it will never break again
  • http://testing.drupal.org can test patches for you before you contribute them! notifies you in 24 hours if patch doesn't work
    
  • Database: the Next Generation (DB:TNG)
    
    • uses PHP Database Objects (PDO)
    • dynamic query builder replaces hook_db_rewrite_sql()
  • FieldAPI (CCK in core)
    • swappable storage back-ends
    • CCK today makes each field its own table only if multiple values or multiple content types
    • FieldAPI makes each field its own table always -- means performance hit
  • Performance
    • registry table holds cache of every function, class, etc. from all modules for faster hook calls, better break-up of code in modules
    • new operation-specific hooks to remove need for $op parameter (also works in D6)
    • new _load_multiple hooks to load multiple items at once
    • file API improvements, image handling in core, native file fields, file management console
  • new hooks in D7
    • allow modules to see which other modules, taxonomy, etc. installed
    • improvements to Aggregator API
    • hook_page_alter() -- modules can alter other modules' generated pages!

Drupal CDN Integration

session details: Drupal CDN Integration

note: speaker is currently developing a Drupal module and synchronization daemon as his masters thesis.

• page rendering performance is included in page loading performance
• why does it matter? because users care! Google found another 500ms caused 20% fewer searches!
• important to Drupal because it powers big, popular sites and is used by many sites around the world
  
• speeding up Drupal would speed up the Internet itself
• 80%+ of load time is static components -- don't require cookies, etc.
• A CDN is a collection of servers in multiple locations serving static content
  • geographical spread
  • can be populated by pull: virtually no setup, but also no flexibility, redundant traffic
    
  • can be populated by push: more setup, but more flexible, no redundant traffic, lock-in to a particular CDN
• The CDN Effect: YSlow penalizes you two letter grades if it thinks you don't have a CDN, even if there is no performance problem
• Episodes is a Javascript tool to measure page loading sequence -- plug-in for Firebug, Drupal module available next week!
• how to integrate Drupal with a CDN?
  
  • change all calls to base_path() function
  • Simple CDN module offers some limited improvement
  • CDN integration module -- not scalable yet!
  • CDN integration will be improved in coming months with push capabilities
  • daemon must be extensible, robust & scalable
• Future goals
  • transparency (transfer protocol irrelevant)
  • mixing CDNs with static file servers
  • processing before sync: image optimization, transcoding media files
    
  • detect new files instantly!
  • daemon could also be used as a massive back-up tool, transcoding server, tool for creating one's own CDN from shared servers
    

A Paradigm for Reusable Drupal Features

session details: A Paradigm for Reusable Drupal Features

• the theory
  • enable the Blog module, each user gets a URL and RSS feed
  • a blog is a "feature," and so is a pressroom, an image gallery, a publication library, etc.
  • more complex example: a pressroom with fields, blocks, theming, etc.
  • 3 problems: slow process, not reusable, disconnected pieces
  • Context module (currently alpha for D6) lets you assign related elements to a single feature which can be scripted or exported (manually)
    
• the practice
  • set up content type, view,
  • views have "spaces" filters to define possible contexts (thanks to Spaces module)
    
  • go to /admin/build/context and select elements that relate to your feature
  • go to /spaces and enable & customize the feature
  • go to /admin/build/menu and select Spaces for primary links
  • To avoid doing this on every site, make feature into its own module (manually)
    
  • combine Spaces with Organic Groups to allow features to look & act different in different parts of the site
    
• the future
  • why put site building in code? because it's reusable, distributable, and versionable
  • but scripts are a pain to debug, maintain, and extend
  • new model: "exportables" -- similar to how we currently export views to code: normal/default/overridden
  • exportables provide definitions, not scripts; are easy to maintain; and are overridable and extensible.
  • views, panels, context_ui, and imagecache all have exportables
  • cck, taxonomy, filter, menu, triggers don't yet have exportables
  • recipe for exportables:
    • string identifiers good, ID numbers bad
    • create API functions for collecting all items at once
    • create a structured array or object to represent your item
    • use hook_item_export in your module
  • wouldn't it be cool if a module let you export all your site's features into a module?
  • to prepare for this day, add exportables to your modules!
    

What's New in Web Development

See session description: What's New in Web Development

not a Drupal-specific session

HTML 5

• new standard is long overdue, so need to standardize what people are already doing in nonstandard ways (Javascript, Flash, etc.)
  
• new structural elements: <section>, <nav>, <article>, <aside>, <footer>, <figure>, <header>, <dialog>
  • these all behave like spans but are used for structuring the content
  • allow nicer CSS with fewer classes
  • can use these tags already in most current browsers, except in IE.  Can define with javascript document.createElement('section');
• new input types for forms, only implemented in Opera so far
  
  • email and url inputs with validation
  • date and time pickers, range slider
  • validation depends on the browser, so that's already problematic
  • required attribute: form will not submit if field is not filled in
  • specs will not be finalized until implemented by two major browsers, to prevent unimplementable specs
• storing data on the client (without sending to server)
  
  • database storage API uses SQLite, separate from cache, cookies, etc.
    
  • local and session storage with key-value pairs (similar to $_SESSION in PHP, but on the client)
  • supported by Mozilla, WebKit (Safari, Chrome, Konqueror), Opera
  • allows the user to customize content of a page regardless of where it is stored, provided they access from the same client
  • security implications are unclear - SQL injection could happen on client side, e.g. from a virus
• <canvas> tag
  • allows an arbitrary 2D drawing with hotspots, etc.
  • already supported by Mozilla, WebKit, Opera
  • can be emulated with VML in IE
  • examples:
    
    • http://code.google.com/p/flot/
    • http://wiki.github.com/sorccu/cufon/about
    • http://ejohn.org/blog/processingjs/
• Canvas Fonts
  • uses <canvas> or VML to render fonts that are not installed on client
  • converts font files to drawings
• Drag and drop now possible without Javascript
• cross-document message exchange
• data grids - like a spreadsheet table - click headers to sort - not yet implemented
• undo manager - allows standard way for users to undo changes they make in Web apps
• Web sockets - standard way to push data to a page already loaded in client
• <video>, <audio> tags for playing media without embedding objects

New browser tools

• Google Gears
  • allows Web applications to work offline such as Gmail, Google Docs
    
  • allows worker pools for compute-intensive applications
  • databases (similar to but not compatible with HTML 5)
  • geolocation capabilities for GPS-enabled devices
• Google Native Client
  • allows any executable (for a given processor and OS) to be run as a plug-in in the browser
  • has major security implications!
• Silverlight
  • Microsoft's Flash clone, based on XAML and WPF
  • implements .NET languages on client

CSS 3

• split up specification into modules so it's not so huge
  
• some parts are already implemented (but not in IE!), so you can use them now -- they degrade gracefully
• border-radius for rounded corners
• user-modify: read-write allows editing on the client
• box-shadow
• background now has opacity
• border-image
• text-overflow: ellipsis
• column-width and column-gap for multi-column layouts
  
• font downloads (beware copyrights)
• WebKit allow tranforms and rotations

Client-side XSLT

• transform XML to multiple formats on the client
• implementation differs by browser
• example: WoW Armory

The Next Decade

session details: The Next Decade

• there are now 4.5 million tweets per day -- people are creating volumes of content on their cell phones
• why is Twitter significant? because it's an example of traffic volume that cannot be handled by a single database, and Drupal is currently limited to a single database (unless you hack core...)
  
• client-server model is obsolete, can't keep up with the scale of modern applications
  • we need to lose the assumption that data needs to be consistent across servers, because it creates bottlenecks
  • consistency is an abstraction anyway; it should be a question, not an assumption
  • immediate consistency is really only important to the originator of the data; doesn't affect others in the same way.
  • what's more important is eventual consistency (5 mins or so)
  • conflicts arise when people change things like tags on one server while others are posting content with those tags on another server
  • anticipate conflicts and build applications to resolve them -- can't expect databases to handle them
• we already have this problem at drupal.org and have mostly resolved it
• can't just assign server IDs to table keys because (theoretically) two people could post at the same instant on different servers -- which one should display first?
• wikis (and Google Docs) offer a model for resolving content conflicts because revisions are essentially patches rather than entire new versions of the content, so conflicts only involve a line or two at a time instead of the whole page ... there is not yet a Drupal module to do this!
  
• everything is possible, but some things are prohibitively difficult!
  

Why I Hate Drupal

(note: I don't hate Drupal; this is the title of a DrupalCon session by James Walker.)  Session details: Why I Hate Drupal

• James ("walkah") Walker has been working full-time on Drupal since 2003
  
• co-founded Bryght, a hosted Drupal service (before Drupal had an easy installer)
• Wordpress was founded at the same time and succeeded wildly while Bryght failed
• now works for Lullabot as director of education, trying to make Drupal make sense
  

1. the scary alien head logo -- what's that about?
   
2. exponential growth ... but Joomla and Wordpress and even SharePoint are way ahead
3. Drupal is a CMS, so should make it easy to manage content, but it is still very difficult for most people
   1. list of modules is daunting and misleading
   2. theming section doesn't offer much improvement; more likely to make it worse
   3. unclear how to upload pictures or format text, and input filters block tags
   4. 4400 contributed modules -- where do you start?
   5. documentation is unclear and hard to search, and community is not supportive of newbies
   6. Views 2 interface is really confusing!
   7. upgrading process is brutal
   8. so upgrade on a staging server... but merging is nearly impossible!
4. so maybe Drupal is a development framework... but limitations of PHP have dictated some weird coding
   1. arrays as objects?
   2. some of our most creative work is justifying our half-assed compromises
   3. we're told not to hack core, but drupal_alter / hook_form_alter essentially build things twice
   4. "Not Invented Here" philosophy sometimes prevents us using external libraries even if they've already solved the problem
      1. PHP has a built-in function for loading configuration files, but we don't use it for .info files!
      2. used to use a 3rd-party XMLRPC library, but we wrote our own
      3. James himself wrote our own OpenID library and now has to support it
   5. frameworks should make our lives easier!  but API is not intuitive at all and many obvious functions are missing
   6. some column names are reused in different contexts in multiple tables, like vid
5. so maybe Drupal is a community... but growing so fast that nobody knows anybody anymore, so it's hard to work effectively
   
   1. to our credit, most original community members are still involved
   2. Drupal 6 had 900 contributors, but only 2 could commit patches -- they're great people, but not superhuman and have to understand all of core
      
   3. anybody can change an issue to "reviewed and tested by the community," so only some testers have credibility, and new contributors don't know whom to pester
   4. not a meritocracy but a do-ocracy ... so "do more" and sleep less and have no life
   5. Drupal is not a movement because only developers are running with it... when non-techies run with it, then maybe it will be a movement
6. need to choose -- what is Drupal trying to be, and to whom?  Core needs to be useful to somebody out of the box.

Closing Session

• 1400 registered, 127 led sessions
• 55 sponsoring companies, 96 individual sponsors, $300,000 total, not counting ticket sales
  
• well over $100,000 profit from this conference, to be applied to the project!
• videos of all sessions will be online by Saturday night -- over 60 online already!
• Paris venue is not as big as DC convention center, but it's old and beautiful, and family events are planned!