Drupal Fire - Quick Roundup from important Drupal blogs and sites
I believe that the "digitalization" of the world is a "megatrend" that will continue for decades. On the one hand, organizations are shifting their businesses online, often inventing new ways to do business. On the other hand, customers are expecting a better and smarter user experience online.
This has led to two important sub-trends: (1) the number of sites an organization is creating and managing is growing at a rapid clip, (2) so is the underlying complexity of each website.
Forrester Research recently surveyed large enterprises about their website portfolio and found that on average they manage 268 properties across various channels. On top of that, each website is becoming more and more advanced. They evolved from simple HTML pages to dynamic websites to digital experience platforms that need to integrate with many other business systems. The combination of these two trends -- increasing number of sites and the growing complexity of each site -- poses real challenges to most organizations.
At Acquia, we are seeing this explosion of websites in the enterprise every day. Many organizations have different websites for different brands and products, want different websites for each country or region they operate in, or offer separate portals for their affiliates, dealers, agents or franchises. We're also seeing organizations, small and large, operate a large number of marketing campaign websites. These organizations aren't focused on scaling back their online properties but rather how best to manage them over time.
I outlined this trend and its challenges almost five years ago (see Acquia product strategy and vision) and most of it is still relevant today, if not more relevant. In this blog post, I want to give you an update and share some lessons learned.
Most larger organizations run many different types of websites. It's not unusual for a small organization to have ten websites and for a large organization to have hundreds of websites. Some of Acquia's largest customers operate thousands of websites.
Most organizations struggle to manage their growing portfolio of digital properties. You'd be surprised how many organizations have more than 20 different content management systems in use. Often this means that different teams are responsible for them and that they are hosted on different hosting environments. It is expensive, creates unnecessary security risks, poses governance challenges, leads to brand inconsistency, makes it difficult to create a unified customer experience, and more. It costs large organizations millions of dollars a year.
Drupal's unfair advantage
When managing many sites, Drupal has an unfair advantage in that it scales from simple to complex easily. That scalability, coupled with a vast ecosystem of modules, elevate Drupal from a single site point solution to a platform on which you can build almost any kind of site: a brand site, a corporate website, a customer support community, a commerce website, an intranet, etc. You name it.
This is in contrast to many of Drupal's competitors that are either point solutions (e.g. SharePoint is mainly used for intranets) or whose complexity and cost don't lend themselves to managing many sites (e.g. Adobe Experience Manager and Sitecore are expensive solutions for a quick marketing campaign site, while WordPress can be challenging for building complex websites). So the first thing people can do is to standardize on Drupal as a platform for all of their site needs.
By standardizing on Drupal, organizations can simplify training, reduce maintenance costs, streamline security, and optimize internal resources – all without sacrificing quality or requirements. Standardizing on Drupal certainly doesn't mean every single site needs to be on Drupal. Transitioning from 20 different systems to 3 still translates into dramatic cost savings.
The Acquia advantage
Once an organization decides to standardize on Drupal, the question is how best to manage all these sites? In 2013 we launched Acquia Cloud Site Factory (ACSF), a scalable enterprise-grade multi-site management platform that helps organizations to easily create, deploy, and govern all their sites. Today, some of Acquia's biggest customers use ACSF to manage hundreds of sites - in fact on average an ACSF customer is currently managing 170 websites within their Site Factory platform and that number is growing rapidly.
Acquia commissioned Forrester Research to analyze the benefits to organizations who have unified their sites on a single platform. Forrester found that moving to a single platform dramatically reduced site development and support costs, conserved IT and marketing resources, and improved standardization, governance, and scalability — all while accelerating time-to-market and the delivery of better digital experiences.
One of the things we've learned is that a complete multi-site management solution needs to include advanced tools for both developers and content managers. The following image illustrates the different layers of a complete multi-site management solution:
The different layers of the Acquia Cloud Site Factory solution stack.
Let's go through these individually from the bottom up.
Consider an organization that currently has 50 websites, and plans to add 10-15 more sites every year. With ACSF these sites run on a platform that is scalable, secure, and highly reliable. This infrastructure also allows hardware resources to be logically isolated based on the site's needs as well as scaled up or down to meet any ad-hoc traffic spikes. These capabilities enable organizations to simplify multi-site management efforts and eliminate operational headaches.
If this organization with 50 sites had individual codebases for each site, that would be 50 disparate codebases to manage. With ACSF, the underlying code can be shared and managed in one central place but the content, configuration, and creative look-and-feel can be catered to each individual sites' needs. ACSF also enable developers to easily add or remove features from their codebases for individual sites. ACSF also comes with tools to automate the process of rolling out updates across all their sites.
Organizations with many sites also need efficient ways to manage and govern them effectively; from developer tools such as Git, Travis, or Behat that enable them to build, test, and maintain sites, to tools for non-developers to quickly clone and spin up sites using site templates defined by a brand manager or a digital design team. ACSF enables customers to effortlessly manage all their sites from a single intuitive dashboard. Developers can create groups of users as well as sites allowing certain users to manage their dedicated domain of sites without stepping over other sites. Non-technical content managers can quickly spin up new sites by cloning existing ones they have access to and updating their configuration, content, and look-and-feel. These features allow organizations to launch sites at unprecedented speed inherently improving their overall time to market.
Finally, I should mention personalization. For a few years now we have been developing Acquia Lift. Acquia Lift builds unified customer profiles across all your websites, and uses that information to deliver real-time, contextual, and personalized experiences. For instance, if the organization in the above example had 50 websites for each of their 50 different products, Acquia Lift can present relevant content to its users as they browse across these different sites. This enables organizations to convert anonymous site visitors into known customers and establish a meaningful engagement between them.
I believe that the "multi-sites era" will continue to accelerate; not only will we see more sites, but every site will become increasingly complex. Organizations need to think about how to efficiently manage their website portfolio. If you're not thinking ahead, you're falling behind.
We've just achieved a big and exciting milestone in Drupal 8's development: starting with Drupal 8 beta 15, we are providing beta-to-beta upgrade paths. This will make it much easier to update Drupal 8 development sites between the current beta and future betas and release candidates.
There has been a lot of excitement building around Drupal 8. Many have been wondering when to start building Drupal 8 sites. The answer for many is NOW.
This change signals an important opportunity for organizations to begin developing with Drupal 8, especially for:
- Sites that rely mainly on the expanded functionality provided by Drupal 8 core alone.
- Projects that will take months of development time.
- Sites for which the benefits of Drupal 8's outweigh the effort needed to port (or work around) contributed modules that do not yet have Drupal 8 versions.
I strongly encourage you to evaluate Drupal 8 for your upcoming projects. Also, if you haven't already, now is the time to port contributed modules so they are ready in time for Drupal 8's release! There are only about five release-blocking issues left before we create the first release candidate.
Note that betas are not supported releases of Drupal, and both developing and launching sites with beta releases present risks. However, I'm pleased that various Drupal agencies, including Acquia, are helping to eliminate those risks through support, development, and hosting optimized for Drupal 8.
Before you get started with Drupal 8, be sure to review all the release notes for beta 15.
Although Drupal tries its hardest to combine CSS and JS in the most optimal way, sometimes it needs some guidance. Using a practical example, we’ll become familiar with AdvAgg’s information tab, letting us identify files within aggregates so that they can receive special processing within other Drupal hooks.
In this week’s REST Easy tutorial, we tackle the process of adding entity references to your API endpoints. Entity references create relationships between two separate data structures in Drupal. Exposing that link within your API is critical to providing a comprehensive content model.
There are rare occasions when you want to re-index all your site's content in Solr. Such occasions include:
Major Drupal version upgrade (e.g. from Drupal 6.x to Drupal 7.x).
Changing your Solr schema to include more search criteria.
Upgrading your Solr server to a new major version.
Moving your Solr server from an old server to a new one.
I've spent a fair amount of time thinking about how to win back the Open Web, but in the case of digital distributors (e.g. closed aggregators like Facebook, Google, Apple, Amazon, Flipboard) superior, push-based user experiences have won the hearts and minds of end users, and enabled them to attract and retain audience in ways that individual publishers on the Open Web currently can't.
In today's world, there is a clear role for both digital distributors and Open Web publishers. Each needs the other to thrive. The Open Web provides distributors content to aggregate, curate and deliver to its users, and distributors provide the Open Web reach in return. The user benefits from this symbiosis, because it's easier to discover relevant content.
As I see it, there are two important observations. First, digital distributors have out-innovated the Open Web in terms of conveniently delivering relevant content; the usability gap between these closed distributors and the Open Web is wide, and won't be overcome without a new disruptive technology. Second, the digital distributors haven't provided the pure profit motives for individual publishers to divest their websites and fully embrace distributors.
However, it begs some interesting questions for the future of the web. What does the rise of digital distributors mean for the Open Web? If distributors become successful in enabling publishers to monetize their content, is there a point at which distributors create enough value for publishers to stop having their own websites? If distributors are capturing market share because of a superior user experience, is there a future technology that could disrupt them? And the ultimate question: who will win, digital distributors or the Open Web?
I see three distinct scenarios that could play out over the next few years, which I'll explore in this post.
This image summarizes different scenarios for the future of the web. Each scenario has a label in the top-left corner which I'll refer to in this blog post. A larger version of this image can be found at http://buytaert.net/sites/buytaert.net/files/images/blog/digital-distrib....
Scenario 1: Digital distributors provide commercial value to publishers (A1 → A3/B3)
Digital distributors provide publishers reach, but without tangible commercial benefits, they risk being perceived as diluting or even destroying value for publishers rather than adding it. Right now, digital distributors are in early, experimental phases of enabling publishers to monetize their content. Facebook's Instant Articles currently lets publishers retain 100 percent of revenue from the ad inventory they sell. Flipboard, in efforts to stave off rivals like Apple News, has experimented with everything from publisher paywalls to native advertising as revenue models. Except much more experimentation with different monetization models and dealmaking between the publishers and digital distributors.
If digital distributors like Facebook succeed in delivering substantial commercial value to the publisher they may fully embrace the distributor model and even divest their own websites' front-end, especially if the publishers could make the vast majority of their revenue from Facebook rather than from their own websites. I'd be interested to see someone model out a business case for that tipping point. I can imagine a future upstart media company either divesting its website completely or starting from scratch to serve content directly to distributors (and being profitable in the process). This would be unfortunate news for the Open Web and would mean that content management systems need to focus primarily on multi-channel publishing, and less on their own presentation layer.
As we have seen from other industries, decoupling production from consumption in the supply-chain can redefine industries. We also know that introduces major risks as it puts a lot of power and control in the hands of a few.
Scenario 2: The Open Web's disruptive innovation happens (A1 → C1/C2)
For the Open Web to win, the next disruptive innovation must focus on narrowing the usability gap with distributors. I've written about a concept called a Personal Information Broker (PIM) in a past post, which could serve as a way to responsibly use customer data to engineer similar personal, contextually relevant experiences on the Open Web. Think of this as unbundling Facebook where you separate the personal information management system from their content aggregation and curation platform, and make that available for everyone on the web to use. First, it would help us to close the user experience gap because you could broker your personal information with every website you visit, and every website could instantly provide you a contextual experience regardless of prior knowledge about you. Second, it would enable the creation of more distributors. I like the idea of a PIM making the era of handful of closed distributors as short as possible. In fact, it's hard to imagine the future of the web without some sort of PIM. In a future post, I'll explore in more detail why the web needs a PIM, and what it may look like.
Scenario 3: Coexistence (A1 → A2/B1/B2)
Finally, in a third combined scenario, neither publishers nor distributors dominate, and both continue to coexist. The Open Web serves as both a content hub for distributors, and successfully uses contextualization to improve the user experience on individual websites.
Right now, since distributors are out-innovating on relevance and discovery, publishers are somewhat at their mercy for traffic. However, a significant enough profit motive to divest websites completely remains to be seen. I can imagine that we'll continue in a coexistence phase for some time, since it's unreasonable to expect either the Open Web or digital distributors to fail. If we work on the next disruptive technology for the Open Web, it's possible that we can shift the pendulum in favor of “open” and narrow the usability gap that exists today. If I were to guess, I'd say that we'll see a move from A1 to B2 in the next 5 years, followed by a move from B2 to C2 over the next 5 to 10 years. Time will tell!
Periodically, there is a complaint that PHP conferences are just "the same old faces". That the PHP community is insular and is just a good ol' boys club, elitist, and so forth.
It's not the first community I've been part of that has had such accusations made against it, so rather than engage in such debates I figured, let's do what any good scientist would do: Look at the data!
It’s easy to add node endpoints to your RESTful API - but there’s more to Drupal than nodes. This week we’ll add an endpoint for a taxonomy vocabulary.
How do you prepare for the inevitable, yet moving target of Drupal 8 when you’re busy with client work? Join Four Kitchens Web Chefs as we take the plunge with a practice group.
In the third installment of REST Easy, our RESTful module tutorial series, we’ll take a look at how to filter your API endpoints for results, a great feature that brings in the power of Entity Field Query for your APIs.
Inlining critical CSS on a dynamic CMS such as Drupal or Wordpress doesn’t have to be a pain. Using the Fourword as an example, we’ll go through the process of generating critical CSS, inlining the file dynamically, and asynchronously loading the remaining CSS aggregates using the AdvAgg module.
This week we’ll continue where we left off after doing the ground work of creating a basic endpoint with the RESTful Drupal module. We skipped one crucial part: the body field. This week’s installment will cover our missing field, which requires extra care when used within a RESTful endpoint.
Meteor + Drupal = Real time Drupal interfaces
Why not combine the two and let Drupal provide Meteor with semantic data with Meteor rendering that data in a reactive way?
Drupal is a pretty strong content manager, allowing you to build robust data models, easily enable content revisioning, and build a publishing workflow complex enough for even the strictest of editorial standards. However, the blistering speed of development on Drupal 8 appears glacial when compared with the rate of invention on the frontend of web development.
With the first party API building tools built into Drupal 8 core, there’s been a lot of talk about building semantic APIs in Drupal; when, how, and why you should or should not. However, a commonly overlooked piece of this picture is how to go about actually consuming these APIs. And as it turns out, consuming an API, even a well designed one, can present a number of challenges. That’s why we created Saucier, a Node.js framework used to quickly create web pages from data that has been consumed from a Drupal API.
As part of Four Kitchens’ ongoing commitment to Drupal 8, I’ve been collaborating on the next generation version of Site Audit, a project for analyzing Drupal sites for best practices. Earlier this year, Site Audit was selected for inclusion in the Drupal Google Summer of Code 2015.
Why Drupal 8 won't ship with REST content negotiation
July 13th, 2015
Some friends on Twitter were alarmed by this Drupal change record:
Accept header based routing got replaced by a query parameter
curl /node/1 -H "Accept: application/hal_json"
The issues leading to this change are too lengthy to capture on Twitter, so I'll give my perspective here.
What was the problem with accept-routing?
- Most critically, not all CDNs and caches support content negotiation (see #2364011)
- A secondary issue that I think deserves more attention is that even when caches support content-negotiation, the cache hit rate suffers.
The poor hit rate arises because web browsers do not simply declare Accept: text/html, they request a bizarre mishmash of media types depending on the specific browser, version, OS, and even what extensions are installed. A quick test on this site discovered roughly 100 unique Accept headers in 24 hours.
But isn't content-negotiation required to have "pure" REST?
- No. Roy Fielding has said "Neither choice has anything to do with REST" (referring to a debate between extensions or query parameters).
- While I don't deny the aesthetic appeal of the Accept header, there are good reasons why Drupal often has to de-prioritize theoretical purity. As mikl wrote on #2364011, "Drupal should be engineered for the web we have, rather than the one we wish we had."
Is ?_format a Drupalism?
The special _format Routing Parameter comes from Symfony, so I think this would be more accurately called a Symfony-ism, if it's an -ism at all. However, given that REST is an architectural style, and not a specific protocol, any API we design will naturally contain some details unique to Drupal.
Can't Varnish normalize the wide variety of Accept headers?
Yes, but if custom normalization routines in Varnish were the only way to have a high-performing cache, then that would be a bad Drupal-ism. It's better to have broad compatibility with current CDNs and caches.
Could a contrib module re-enable content negotiation?
Probably. The Symfony route filters are plug-ins that can be swapped out.
What alternatives were considered?
Some other options were moving REST resources to an /api path, or using extensions like /node/1.json. In the end, the query parameter won out because it had the least impact on the existing routing system.
I think an ideal solution to both problems would be widespread support for the Alternates: header (RFC 2295). In this way a server can specify what alternate media formats, languages, or encodings are available, and content negotiation can be efficiently performed at the network's edge. Sadly, this RFC appears largely abandoned since it's introduction in 1998, so perhaps the best we can hope for is by the time Drupal 9 is ready, CDNs and browsers will have cleaned up their act.
Acquia has just announced their active support for Drupal 8 on their hosting platform. As Acquia Partners, we’ve been hard at work preparing for the upcoming Drupal 8 release. Four Kitchens will continue to lead and innovate in the Drupal 8 era. In fact, our work in Drupal 8 has already begun - continue reading to learn more!
I'm excited to announce that starting today, Acquia is announcing we're ready to fully support our customers with Drupal 8. This means our professional services, our support, our product engineering, our cloud services … the entire company is ready to help anyone with Drupal 8 starting today.
While Drupal 8 is not yet released (as it has always been said, Drupal 8 will be "ready when it's ready"), the list of release blockers is dwindling ever closer to zero, and a beta-to-beta upgrade path will soon be provided in core. These factors, along with Acquia's amazing team of more than 150 Drupal experts (including a dedicated Drupal 8 engineering team that has contributed to fixing more than 1,200 Drupal 8 issues), gives us full confidence that we can make our customers successful with Drupal 8 starting today.
In the process of working with customers on their Drupal 8 projects, we will contribute Drupal 8 core patches, port modules, help improve Drupal 8's performance and more.
I'm excited about this milestone, as Drupal 8 will be a truly ground-breaking release. I'm most excited about the architectural enhancements that strongly position Drupal 8 for what I've called the Big reverse of the Web. For the web to reach its full potential, it will go through a massive re-platforming. From Flipboard to the upcoming release of Apple News, it's clear that the web is advancing into the “post-browser” era, where more and more content is "pushed" to you by smart aggregators. In this world, the traditional end-point of the browser and website become less relevant, requiring a new approach that increases the importance of structured content, metadata and advanced caching. With Drupal 8, we've built an API-driven architecture that is well suited to this new “content as a service” approach, and Drupal 8 is ahead of competitive offerings that still treat content as pages. Check out my DrupalCon Los Angeles keynote for more details.
Drupal 8 packs a historic amount of site building features which make producing websites easier than ever with core or just a couple contributed modules only. There are already various live Drupal 8 multilingual sites using little more but core.
It is hard to grasp the many things with useful levers and knobs in Drupal 8. Think about combining views with entity view modes and blocks; block language visibility with menus; user preferences with comment submission; language filtering and entity rendering; translatable fields with administration views; and so on and on.
Wouldn't it be fun to experiment with the possibilities and come up with clever ways to combine core features to solve common problems? You may be familiar with the name and format of O'Reilly's Hacks Series which reclaims the term "hacking" for the good guysfolks — innovators who explore and experiment, unearth shortcuts, create useful tools, and come up with fun things to try on their own. The excellent series inspired the name and format of our contest.
Long story short, hereby, we announce the Drupal 8 multilingual site building hacks contest!
- Come up with clever ways to combine Drupal 8 core features (and if needed one or at most two contributed modules) to fulfill a multilingual site building need.
- Write up the steps taken. See an example in hack #1. (We'll do light editing of the post if needed, don't let perfection be the enemy of good).
- Register on http://drupal8multilingual.org/user to submit entries (requires approval for spam protection).
- Submit entries by end of day (CEST) July 31st.
- One person may submit as many entries as they wish.
- All entries will be published after review (and possible light editing).
What is in it for you?
The top 3 best hacks will receive unique presents from Hook42 and Amazee Labs! (Further sponsors welcome). You'll either receive the presents at DrupalCon Barcelona or we'll mail it to you if you are not coming to DrupalCon. This is of course additionally to the joy of getting to play with some of the less frequented but definitely no less fun features of Drupal 8.
What is in it for us?
All hacks will be published under Creative Commons Attribution-ShareAlike 4.0, so the community will benefit. Additionally to that Gábor Hojtsy and Vijayachandran Mani are building an open source presentation with the best tips (same license). This will be presented at Drupalaton Hungary and DrupalCon Barcelona. Similar to our existing open source workshop, everyone will be able to present this at local meetups and camps or follow along at home at their own pace.
What kind of hacks are we looking for?
Hack #1 is hopefully a good example. Really the only common thread between the hacks would be to satisfy a multilingual site need or use multilingual features in some other clever way (even for features that are not necessarily multilingual). Some ideas for hacks that may help you start off experimenting:
- Swap textual site logo Need to swap a site logo with text on it for different languages? Use a translatable custom block with an image field. Configure the display mode and add some custom CSS if needed.
- Translator todo helper Create a views block for content translators to summarize the number of outdated translations they have to update (and link to content administration filtered to that language)
- Language dependent front page Use block visibility to display up to date content on a well maintained language while an About us / Contact us page on languages where resources are limited to maintain useful fresh content.
Of course these are just some things we made up (although still eligible for the contest). Looking for your creative ideas and solutions!
Questions, concerns? Contact us!
This is a crosspost from http://www.drupal8multilingual.org/hacks.