Lullabot: Decoupled Drupal Hard Problems: Image Styles

Planet Drupal - 26. Oktober 2017 - 0:52

As part of the API-First Drupal initiative, and the Contenta CMS community effort, we have come up with a solution for using Drupal image styles in a decoupled setup. Here is an overview of the problems we sought to solve:

  • Image styles are tied to the designs of the consumer, therefore belonging to the front-end. However, there are technical limitations in the front-end that make it impossible to handle them there.
  • Our HTTP API serves an unknown number of consumers, but we don't want to expose all image styles to all consumers for all images. Therefore, consumers need to declare their needs when making API requests.
  • The Consumers and Consumer Image Styles modules can solve these issues, but it requires some configuration from the consumer development team.
Image Styles Are Great

Drupal developers are used to the concept of image styles (aka image derivatives, image cache, resized images, etc.). We use them all the time because they are a way to optimize performance on our Drupal-rendered web pages. At the theme layer, the render system will detect the configuration on the image size and will crop it appropriately if the design requires it. We can do this because the back-end is informed of how the image is presented.

In addition to this, Drupal adds a token to the image style URLs. With that token, the Drupal server is saying I know your design needs this image style, so I approve the use of it. This is needed to avoid a malicious user to fill up our disk by manually requesting all the combinations of images and image styles. With this protection, only the combinations that are in our designs will be possible because Drupal is giving a seal of approval. This is transparent to us so our server is protected without even realizing this was a risk.

The monolithic architecture allows us to have the back-end informed about the design. We can take advantage of that situation to provide advanced features.

The Problem

In a decoupled application your back-end service and your front-end consumer are separated. Your back-end serves your content, and your front-end consumer displays and modifies it. Back-end and front-end live in different stacks and are independent of each other. In fact, you may be running a back-end that exposes a public API without knowing which consumers are using that content or how they are using it.

In this situation, we can see how our back-end doesn't know anything about the front-end(s) design(s). Therefore we cannot take advantage of the situation like we could in the monolithic solution.

The most intuitive solution would be to output all the image styles available when requesting images via JSON API (or REST core). This will only work if we have a small set of consumers of our API and we can know the designs for those. Imagine that our API serves to three, and only three, consumers A, B and C. If we did that, then when requesting an image from consumer A we would output all the variations for all the image styles for all the consumers. If each consumer has 10 - 15 image styles, that means 30 - 45 image styles URLs, where only one will be used.

undefined

This situation is not ideal because a malicious user can still generate 45 images in our disk for each image available in our content. Additionally, if we consider adding more consumers to our digital experience we risk making this problem worse. Moreover, we don't want the presentation from one consumer sipping through another consumer. Finally, if we can't know the designs for all our consumers, then this solution is not even on the table because we don't know what image styles we need to add to our back-end.

On top of all these problems regarding the separation of concerns of front-end and back-end, there are several technical limitations to overcome. In the particular case of image styles, if we were to process the raw images in the consumer we would need:

  • An application runner able to do these operations. The browser is capable of this, but other more challenged devices won't.
  • A powerful hardware to compute image manipulations. APIs often serve content to hardware with low resources.
  • A high bandwidth environment. We would need to serve a very high-resolution image every time, even if the consumer will resize it to 100 x 100 pixels.

Given all these, we decided that this task was best suited for a server-side technology.

In order to solve this problem as part of the API-First initiative, we want a generic solution that works even in the worst case scenario. This scenario is an API served by Drupal that serves an unknown number of 3rd party applications over which we don't have any control.

How We Solved It

After some research about how other systems tackle this, we established that we need a way for consumers to declare their presentation dependencies. In particular, we want to provide a way to express the image styles that consumer developers want for their application. The requests issued by an iOS application will carry a token that identifies the consumer where the HTTP request originated. That way the back-end server knows to select the image styles associated with that consumer.

undefined

For this solution, we developed two different contributed modules: Consumers, and Consumer Image Styles.

The Consumers Project

Imagine for a moment that we are running Facebook's back-end. We defined the data model, we have created a web service to expose the information, and now we are ready to expose that API to the world. The intention is that any developer can join Facebook and register an application. In that application record, the developer does some configuration and tweaks some features so the back-end service can interact optimally with the registered application. As the manager of Facebook's web services, we are not to take special request from any of the possible applications. In fact, we don't even know which applications integrate with our service.

The Consumers module aims to replicate this feature. It is a centralized place where other modules can require information about the consumers. The front-end development teams of each consumer are responsible for providing that information.

This module adds an entity type called Consumer. Other modules can add fields to this entity type with the information they want to gather about the consumer. For instance:

  • The Consumer Image Styles module adds a field that allows consumer developers to list all the image styles their application needs.
  • Other modules could add fields related to authentication, like OAuth 2.0.
  • Other could gather information for analytic purposes.
  • Maybe even configuration to integrate with other 3rd party platforms, etc.
The Consumer Image Styles Project

Internally, the Consumers module takes a request containing the consumer ID and returns the consumer entity. That entity contains the list of image styles needed by that consumer. Using that list of image styles Consumer Image Styles integrates with the JSON API module and adds the URLs for the image after applying those styles. These URLs are added to the response, in the meta section of the file resource. The Consumers project page describes how to provide the consumer ID in your request.

{ "data": { "type": "files", "id": "3802d937-d4e9-429a-a524-85993a84c3ed" "attributes": { … }, "relationships": { … }, "links": { … }, "meta": { "derivatives": { "200x200": "https://cms.contentacms.io/sites/default/files/styles/200x200/public/boyFYUN8.png?itok=Pbmn7Tyt", "800x600": "https://cms.contentacms.io/sites/default/files/styles/800x600/public/boyFYUN8.png?itok=Pbmn7Tyt" } } } }

To do that, Consumer Image Styles adds an additional normalizer for the image files. This normalizer adds the meta section with the image style URLs.

Conclusion

We recommend having a strict separation between the back-end and the front-end in a decoupled architecture. However, there are some specific problems, like image styles, where the server needs to have some knowledge about the consumer. In these very few occasions the server should not implement special logic for any particular consumer. Instead, we should have the consumers add their configuration to the server.

The Consumers project will help you provide a unified way for app developers to include this information in the server. Consumer Image Styles and OAuth 2.0 are good examples where that is necessary, and examples on how to implement it.

Further Your Understanding

If you are interested in alternative ways to deal with image derivatives in a decoupled architecture. There are other alternatives that may incur extra costs, but still worth checking: Cloudinary, Akamai Image Converter, and Origami.

Hero Image by Sadman Sakib

Drupal Commerce: Beta release for Commerce Discount 7.x-1.0

Planet Drupal - 25. Oktober 2017 - 23:30

Commerce Discount improves Commerce 1.x by providing a custom entity type for managing Product and Order level discounts, including more complicated discounts like free shipping upgrades and BOGO offers. The module makes it easier for merchants to create promotions that would otherwise require the use of the Rules UI or even custom code, tasks that are similarly beyond the reach of most casual Drupal users.

Even as we've worked to improve the user experience even further in Commerce 2.x by making Promotions a core module, we continue to work to do to improve the experience for 1.x users. Today, after a month of focused contrib time at Commerce Guys team and review from end users like Thomas Jonas at the University of Minnesota, we're proud to announce the release of a long overdue beta version for the module.

Mediacurrent: DrupalCamp Atlanta 2017 Highlights

Planet Drupal - 25. Oktober 2017 - 22:32

It's official: the countdown to DrupalCamp Atlanta is on. In just two weeks (November 2 - November 4), Mediacurrent will proudly sponsor another great camp in Buckhead, the tech center of ATL. Known for being a top Drupal event in the southeast, DCATL isn't one to miss. It's not too late to register!

Bay Area Drupal Camp: BADCamp videos now available on the website!

Planet Drupal - 25. Oktober 2017 - 22:25
BADCamp videos now available on the website! Grace Lovelace Wed, 10/25/2017 - 1:25pm

Thank you! We had so much fun with all of you at BADCamp that we're already excited for next year!

Review what you learned and see what you missed!

Are there sessions you weren't able to attend at BADCamp this year? Or maybe you're back at work ready to apply what you learned and wishing you had better notes? Never fear! We took video of the slides from each presentation at BADCamp that includes audio from our expert speakers! Just visit our event schedule and click on the sessions you'd like to view. Videos are posted at the top of each session page. 

Share your feedback.

Please take a moment to let us know what you thought about BADCamp—it's just a few questions and will help us improve our future events.

Send Your Feedback

Join us at next year's BADCamp, October 24th through 27th, 2018! 

BADCamp Organizing Collective

Drupal Planet

Geheimer Datenzugriff für US-Behörden eingeschränkt – Microsoft zieht Klage zurück

heise online Newsticker - 25. Oktober 2017 - 21:00
US-Behörden sollen zwar weiterhin im Geheimen auf Kundendaten zugreifen können, die Anbieter dürfen die betroffenen Nutzer aber künftig zeitnah informieren. Microsoft verbucht das als Erfolg und will eine Klage gegen das US-Justizministerium zurückziehen.

Elektroautos: Neun Autos je Ladepunkt

heise online Newsticker - 25. Oktober 2017 - 19:00
Knapp 11.000 Ladepunkte zählte der Bundesverband der Energie- und Wasserwirtschaft in Deutschland. Er meint, die Automobilindustrie müsse nun nachziehen.

Zahlen, bitte! Für 60.300 Dollar mit Überschall über den Atlantik

heise online Newsticker - 25. Oktober 2017 - 19:00
Am 24. Oktober 2003 startete das Überschallflugzeug Concorde zu seinem letzten Flug von New York nach London. Damit ging die Ära des Überschallflugs zu Ende – vorerst.

DUHK: Zufallszahlengenerator ermöglicht Abhör-Attacke auf zehntausende Geräte

heise online Newsticker - 25. Oktober 2017 - 18:30
Mehr als 25.000 übers Internet erreichbare Fortinet-Geräte sind anfällig für passive Lauschangriffe gegen verschlüsselte Verbindungen. Verantwortlich ist fehlender Zufall.

Kartellamt nimmt Vergleichsportale ins Visier

heise online Newsticker - 25. Oktober 2017 - 18:30
Die Wettbewerbshüter wollen mit einer Untersuchung der Vergleichsportale mögliche Probleme aufdecken und für mehr Transparenz sorgen. Verbraucher müssen sich auf die Angebote verlassen können, meint der Kartellamtschef.

Elevated Third: Lessons Learned: Component Based Design with Paragraphs

Planet Drupal - 25. Oktober 2017 - 18:04
Lessons Learned: Component Based Design with Paragraphs Lessons Learned: Component Based Design with Paragraphs Anthony Simone Wed, 10/25/2017 - 10:04

 

 

 

The ideas of Atomic Design and component based design allow one to create an established structure within which a large scale front end project can be built. The CMS space hasn’t always been the most friendly toward implementing these types of patterns. Whether it’s difficulty in creating a content architecture that models your front end design system within Drupal or the feeling of lack of control over generated markup, it can feel like an uphill battle.

The Paragraphs module gives us the tools to create much more well defined and structured component based architectures upon which modular front end systems can be built. The Paragraphs module, however, comes with no rules. As a site architect and front end developer, you must decide how to implement Paragraphs. There is definitely a lot of room for flexibility in implementation, but there are many best practices that can be followed to allow for a very clean, scalable, and extendable front end design system to be built within Drupal 8.

The goals of this session will be the following:

  • Review the basic concepts and benefits of component based design
  • Discuss the paragraphs module and how to create an implementation based on a well defined content architecture 
  • Explore some Drupal best practices that allow for a successful component based design system implementation

Display-Drama bei Google: Pixel 2 und Pixel 2 XL im Labor

heise online Newsticker - 25. Oktober 2017 - 18:00
Derzeit häuft sich die Kritik an den Displays der neuen Google-Smartphone. Vor allem das Pixel 2 XL sorgt für Unmut: Nutzer klagen über flaue Farben und Einbrenneffekte. Wir sind dem nachgegangen.

AMDs Radeon-Treiber 17.10.2 zum Download: Bestleistung für Wolfenstein 2, Destiny 2 & AC Origins

heise online Newsticker - 25. Oktober 2017 - 18:00
Der neue AMD-Grafikkartentreiber 17.10.2 macht Radeons für Windows 10 1709 fit, ist für die neuesten Spiele optimiert und unterstützt fürs Ethereum-Schürfen bis zu 12 Grafikkarten pro System.

Zwölf Großstädte wollen ab 2025 nur noch emissionsfreie Busse kaufen

heise online Newsticker - 25. Oktober 2017 - 18:00
Die Luftverschmutzung macht den Städten zu schaffen. Zwölf große unter ihnen haben nun gemeinsam versprochen, sie mit konkreten Schritten zu bekämpfen.

App-Stores: Apple App Store und Google Play auf Rekordhoch bei App-Einnahmen

heise online Newsticker - 25. Oktober 2017 - 17:30
Im dritten Quartal 2017 erreichen Google Play und Apple App Store Rekordzahlen bei den App-Downloads und den Umsätzen. Google führt weiterhin beim Downloadwachstum für Apps, iOS-Apps bringen dafür mehr Umsatz.

Android-Schädling Lokibot ist eine Transformer-Malware

heise online Newsticker - 25. Oktober 2017 - 17:00
In erster Linie ist Lokibot auf Bankdaten aus. Wer gegen den Trojaner vorgeht, bekommt ein anderes Gesicht des Schädlings zu sehen und sieht sich mit Erpressung konfrontiert.

Wegen Verschlüsselung: FBI kann auf Tausende Mobilgeräte nicht zugreifen

heise online Newsticker - 25. Oktober 2017 - 17:00
Die US-Bundespolizei kann auf die Hälfte aller beschlagnahmten Mobilgeräte nicht zugreifen, weil die gespeicherten Daten verschlüsselt sind. Die Zahlen des FBI-Chefs könnten nun eine neue Runde in den Crypto Wars einläuten.

Acquia Lightning Blog: Lightning migration to core media

Planet Drupal - 25. Oktober 2017 - 16:57
Lightning migration to core media Adam Balsam Wed, 10/25/2017 - 10:57

It's here! Lightning 2.2.1 provides a migration to the core media system that was introduced in Drupal 8.4.0.

This is a major milestone for us. One of the big advantages of using Lightning over vanilla Drupal or a roll-your-own solution is that as underlying modules evolve, Lightning maintains an update/migration path. This effectively creates a facade in front of media, workflow, and layout functionality. That functionality remains stable no matter what. Of course, this is in addition to the fact that Lightning provides all of that functionality out of the box. (Even though Media is now a part of core, it still doesn't provide the out of the box configuration, experience, and add-ons that Lightning does.)

Core Media migration was #2 in our list of major migrations. It was preceded by a migration from Layout Plugin to the core Layout Discovery module. Next up is Workflow which will involve migrating from Workbench Moderation to core's Workflows and Content Moderation modules.

Special thanks to phenaproxima who is at the intersection of the core, contrib, and Lightning work. To say the migration wouldn't have been possible without him is an understatement.

Want to try it out?

Update your existing codebase:

composer update acquia/lightning --with-dependencies composer update drupal/core

Then check out our 2.2.0 -> 2.2.1 update instructions.

Or build a fresh codebase:

composer create-project acquia/lightning-project MY_PROJECT

 

Netflix will sich 1,6 Milliarden US-Dollar am Finanzmarkt holen

heise online Newsticker - 25. Oktober 2017 - 16:30
Der Streaming-Anbieter will seinen 109 Millionen Kunden exklusive Produktionen bieten. Das kostet Geld.

Landgericht Hamburg distanziert sich von eigener Rechtsprechung zur Linkhaftung

heise online Newsticker - 25. Oktober 2017 - 16:30
Das Landgericht (LG) Hamburg hält nicht länger an einem strengen Haftungsmaßstab bei Verlinkungen auf urheberrechtsverletzende Inhalte fest. Linksetzende können sich nun in bestimmten Fällen auf die Unzumutbarkeit von Nachforschungsmaßnahmen berufen.

Parallelprogrammierung: Mehr Vortragsvorschläge für parallel 2018 sind willkommen

heise online Newsticker - 25. Oktober 2017 - 16:00
Der Call for Proposals für die im März 2018 tagende Softwarekonferenz zur Parallelprogrammierung geht nun bis 12. November.