USA und Großbritannien warnen vor globaler russischer Cyberattacke

heise online Newsticker - 18. April 2018 - 14:00
Schon wieder Russland? Die USA und Großbritannien werfen Moskau vor, in großem Umfang Netzwerkinfrastrukturen zu infiltrieren. Ein Sprecher des Weißen Hauses spricht von einer "gewaltigen Waffe". Betroffen ist demnach einmal mehr Ciscos Smart Install.

Hacking Humanity: "Nur Maschinen können Maschinen regulieren"

heise online Newsticker - 18. April 2018 - 14:00
Ramzi Rizk, Mitgründer der Foto-Plattform EyeEm, hält technische Entwicklungen wie Künstliche Intelligenz für so komplex, dass der Mensch ihnen kein enges Regelkorsett vorschreiben sollte. Die letzte Verantwortung liege trotzdem bei den Programmierern.

Umsatzschwund bei kleinen stationären Einzelhändlern, Online-Handel weiter im Aufwind

heise online Newsticker - 18. April 2018 - 12:30
Keine guten Nachrichten für kleine stationäre Einzelhändler: Einer Branchenumfrage des Handelsverbands Deutschland zufolge rechnen sie 2018 mehrheitlich mit einem Umsatzrückgang. Der Online-Handel boomt dagegen.

Leistungssprung in der Akku-Technik: Silizium-Anoden für BMW-Elektroautos

heise online Newsticker - 18. April 2018 - 12:30
Materialforscher suchen intensiv nach Möglichkeiten, die Leistung von Akkus zu erhöhen. Mit als Erstes könnten Anoden auf der Basis spezieller Silizium-Partikel den Weg in die Praxis finden – unter anderem bei BMW.

Bitcoin-Fork: Online-Wallet-Inhaber gehen leer aus

heise online Newsticker - 18. April 2018 - 12:00
Beim kombinierten Hard-Fork von Bitcoin und Z-Classic zu Bitcoin Private bekamen Besitzer beider Währungen ihr aktuelles Guthaben als Bitcoin Private geschenkt. Wer jedoch sein Kryptogeld in Online-Wallets aufbewahrt, geht meist leer aus.

Werkstattberichte: Neues aus den Fablabs und der Makerszene

heise online Newsticker - 18. April 2018 - 12:00
Wo enstehen gerade neue Makerspaces, welche Termine stehen an und was ist sonst noch los in den Fablabs? Die Werkstattberichte dieses Mal mit Aktionstagen vom Arduino bis zu Nachhaltigkeit.

Neue Sicherheitsfunktionen für Intel-Prozessoren

heise online Newsticker - 18. April 2018 - 12:00
Threat Detection Technology spannt den integrierten Grafikprozessor in RAM-Scans ein, Security Essentials vereinheitlichen Schutzfunktionen verschiedener CPU-Familien.

Fingerabdruck aus Handyfotos rekonstruiert – Polizei schnappt Drogenhändler

heise online Newsticker - 18. April 2018 - 12:00
Aus einem Foto mit Teilen einer Hand konnte die britische Polizei Fingerabdrücke eines Drogenhändlers rekonstruieren und ihn damit überführen.

Dropsolid: Varnish for Drupal 8

Planet Drupal - 18. April 2018 - 12:00
18 Apr Varnish for Drupal 8: the need for speed Niels A Drupal Drupal 8

Our team had been using Varnish a long time for our Dropsolid Drupal 7 project, and we thought the time had come to get it working for Drupal 8 as well. That is why our CTO, Nick Veenhof, organized a meetup about Caching & Purging in Drupal 8. Niels van Mourik gave an elaborate presentation about the Purge module and how it works.
I definitely recommend watching the video and the slides on his blog. In this blog post, we’ll elaborate and build on what Niels explained to us that day. 
First, let’s start off with a quick crash course on what Varnish actually is and how it can benefit your website.

 

Varnish 101

“Varnish Cache is a web application accelerator also known as a caching HTTP reverse proxy. You install it in front of any server that speaks HTTP and configure it to cache the contents. Varnish Cache is really, really fast. It typically speeds up delivery with a factor of 300 - 1000x, depending on your architecture.” (Source: Varnish-cache.org)

In layman’s terms, Varnish will serve a webpage from its own internal cache if it has it available. This drastically reduces the number of requests to the webserver where your application is hosted. This will, in turn, free up resources on your webserver, so your web application can handle more complicated tasks and more users.

In short, Varnish will make your web application faster and will allow you to scale it more efficiently.


How we use Varnish and Drupal 7

How did things typically work in D7? Well, you’d put a Varnish server with a Drupal compatible Varnish configuration file (vcl) in front of your Drupal 7 site and it would start caching it right away - depending, of course, on what is in the vcl and the headers your Drupal site sent.
Next step would be to install the Varnish module from Drupal.org. This module’s sole purpose is to invalidate your Varnish cache and it does so using telnet. This also requires the Varnish server to be accessible from the Drupal backend. This isn’t always an ideal scenario, certainly not when multiple sites are being served from the same Varnish.

The biggest issue when using Drupal 7 with the Varnish module is that invalidation of content just isn’t smart enough. For instance, if you would update one news item you’d only want that page and the ones where the news item is visible to be removed from Varnish’s cache. But that isn’t possible. This isn’t the module’s fault at all - it’s simply the way Drupal 7 was built. There are a few alternatives that do make it a little smarter, but these solutions aren’t foolproof either.

Luckily, Drupal 8 is a whole new ballgame!


How we use Varnish with Drupal 8

Drupal 8 in itself is very smart at caching and it revolves around the three following main pillars (explained from a page cache perspective, it will also cache parts of pages):

  • Cache tags: A page will get a list of tags based on the content that is on it. For instance if you have a news overview, all rendered news items will be added as tags to that page. Allowing you to invalidate that cache if one of those news items change. 
  • Cache context: A page can be different based based on variables from the current request. For instance if you have a news overview that filters out the news items based on a query parameter. 
  • Cache max-age: A page can be served from cache for X amount of time. After the time has passed it needs to be built up again. 

You can read more about Drupal 8’s new caching system here.


All about invalidation

Niels van Mourik created a module called Purge. This is a modular external cache invalidation framework. It leverages Drupal’s cache system to provide easy access to the cache data so you only need to focus on the communication with an external service. It already has a lot of third-party integrations available like Acquia Purge, Akamai and Varnish Purge.

We are now adding another one to the list: the Dropsolid Purge module.


Why does Dropsolid Purge do and why do I need it?

The Dropsolid Purge module enables you to invalidate caches in multiple Varnish load balancers. It also lets you cache multiple web applications by the same Varnish server. The module was very heavily inspired by the Acquia purge module and we reused a lot of the initial code, because it has a smart way of handling the invalidation through tags, but we’ll get in to that a little later. The problem with the Acquia purge module is that it is designed to work on Acquia Cloud, because it depends on certain environment variables and the Varnish configuration is proprietary knowledge of Acquia. This means that it isn’t usable on another environments. 

We also experimented with the Varnish purge module, but it lacked support for cache invalidation in case you have multiple sites/multisites cached by a single Varnish server. This is because the module actually doesn’t tell Varnish which site it should invalidate pages for, so it just invalidates pages for all the sites. It also doesn’t have the most efficient way of passing along the invalidation requests. It contains two ways of sending invalidation request to Varnish: one by one or bundled together. The one by one option results in a lot of requests if you know that updating a single node could easily invalidate 30 tags. Using the Bundle purger could get you to reach the limit of your header size, but more on that later.


What's in the bag?

Currently we provide the following features:

  • Support for tag invalidation and everything invalidation,
  • The module will only purge tags for the current site by using the X-Dropsolid-Site header,
  • The current site is defined by the name you set in config and the subsite directory,
  • Support for multiple load balancers,
  • There is also a default vcl in the examples folder that contains the logic for the bans.

It can be used for any environment if you just follow the installation instructions in the readme.


Under the hood Preparing and handling the responses for/by Varnish

By default, the module will add two headers to every response it gives:

  • X-Dropsolid-Site: A unique site identifier as a hash based on config (you provide through settings.php) and site parameters:
    • A site name
    • A site environment
    • A site group
    • The path of your site (e.g. sites/default or sites/somesubsite)
  • X-Dropsolid-Purge-Tags: A hashed version of each cache tag on the current page. (hashed to keep the length low and avoid hitting the maximum size of the header)

When the response reaches Varnish, it will save those headers along with the cache object. This will allow us to target these specific cache objects for invalidation.

In our vcl file we also strip those headers, so they aren’t visible by the end user:

sub vcl_deliver { unset resp.http.X-Dropsolid-Purge-Tags; unset resp.http.X-Dropsolid-Site; }
Invalidation of pages

For Drupal 8 we no longer use telnet to communicate with Varnish, but we use a BAN request instead. This request will get sent from our site, and it will only be accepted when it comes from our site.  We currently do this by validating the IP of the request against a list of IPs that are allowed to do BAN requests. 

As we mentioned earlier, we provide two ways of invalidating cached pages in Varnish:

  • Tag invalidation: We invalidate pages which have the same cache tags as we send in our BAN request to Varnish.
  • Everything invalidation: We invalidate all pages which are from a certain site.

Tag invalidation

Just like the Acquia purge module, we send a BAN request which contains a group of 12 hashed cache tags which then will be compared to what Varnish has saved. We also pass along the unique site identifier so we indicate we only want to invalidate for a specific site.

Our BAN request has the following headers:

  • X-Dropsolid-Purge: Unique site identifier
  • X-Dropsolid-Purge-Tags: 12 hashed tags

When Varnish picks up this request, it will go through the following logic:

sub vcl_recv { # Only allow BAN requests from IP addresses in the 'purge' ACL. if (req.method == "BAN") { # Same ACL check as above: if (!client.ip ~ purge) { return (synth(403, "Not allowed.")); } # Logic for banning based on tags # https://Varnish-cache.org/docs/trunk/reference/vcl.html#vcl-7-ban if (req.http.X-Dropsolid-Purge-Tags) { # Add bans for tags but only for the current site requesting the ban ban("obj.http.X-Dropsolid-Purge-Tags ~ " + req.http.X-Dropsolid-Purge-Tags + " && obj.http.X-Dropsolid-Site == " + req.http.X-Dropsolid-Purge); return (synth(200, "Ban added.")); } }

We check if the request comes from an IP that is whitelisted. We then add bans for every cache object that matches our unique site identifier and matches at least one of the cache tags we sent along. 

You can easily test this by updating a node and seeing that Varnish will be serving you a new version of its page. 
 

Everything invalidation

When the everything invalidation is triggered, a BAN request is sent with the following headers:

  • X-Dropsolid-Purge-All: True
  • X-Dropsolid-Purge: Unique site identifier

And we execute the following logic on Varnish’s side:
 

sub vcl_recv { # Only allow BAN requests from IP addresses in the 'purge' ACL. if (req.method == "BAN") { # Same ACL check as above: if (!client.ip ~ purge) { return (synth(403, "Not allowed.")); } # Logic for banning everything if (req.http.X-Dropsolid-Purge-All) { # Add bans for the whole site ban("obj.http.X-Dropsolid-Site == " + req.http.X-Dropsolid-Purge); return (synth(200, "Ban added.")); } } }

When Varnish receives a BAN request with the X-Dropsolid-Purge-All header, it will ban all cache object that have the same unique site identifier. You can easily test this by executing the following command: drush cache-rebuild-external.

Beware: a normal drush cache-rebuild will not invalidate an external cache like Varnish.


Why this matters

To us, this is yet another step in making our cache smarter, our web applications faster and our servers leaner. If you have any questions about this post, you can always leave a comment in the comment section below or open an issue on drupal.org.

Are you looking for a partner that will help you to speed up your site, without having to switch hosting? The Dropsolid Platform helps you to adjust and streamline your development processes, without the typical vendor lock-in of traditional hosting solutions. At Dropsolid, we also offer dedicated hosting, but we never enforce our own platform. Dropsolid helps you to grow your digital business - from every possible angle!

 

Blog overview     Get in touch

US-Regierung verbietet Exporte an ZTE

heise online Newsticker - 18. April 2018 - 11:30
Der chinesische Telekom-Konzern ZTE hat nach Angaben der US-Regierung gegen Auflagen verstoßen, die nach dem illegalen Technikexport an Nordkorea und den Iran verhängt worden waren.

Sega: Mega Drive Mini und Shenmue Remastered angekündigt

heise online Newsticker - 18. April 2018 - 11:30
Pünktlich zum 30. Geburtstag des Mega Drive will Sega eine geschrumpfte Neuauflage veröffentlichen. Die Klassiker Shenmue I und II sollen außerdem als Remastered-Version auf aktuelle Plattformen kommen.

Black Shark: Gaming-Smartphone von Xiaomi

heise online Newsticker - 18. April 2018 - 11:00
Das Black Shark von Xiaomi richtet sich speziell an Gamer. An Bord sind ein Snapdragon 845 mit Heatpipe sowie ein separater Bildverarbeitungschip für die Smartphone-Kameras.

Celebgate: Nacktfotos aus iCloud gephisht - vierter Verklagter bekennt sich schuldig

heise online Newsticker - 18. April 2018 - 11:00
Im Fall “Celebgate” hat sich ein weiterer Mann für schuldig bekannt, per Phishing Zugriff auf mehrere Hundert iCloud-Accounts erschlichen und daraus unter anderem Fotos kopiert zu haben – das Material sei zudem mit Dritten getauscht worden.

Chinesische Plattform Weibo will homosexuelle Inhalte doch zulassen

heise online Newsticker - 18. April 2018 - 11:00
Im Rahmen einer Säuberungskampagne wollte der chinesische Kurznachrichtendienst auch "homosexuelle Inhalte" entfernen. Davon nimmt die Plattform nach Protesten Abstand.

Google-KI beherrscht selektives Hören

heise online Newsticker - 18. April 2018 - 11:00
Softwareentwickler bei Google haben ein Verfahren entwickelt, das aus der Tonspur eines Videos die Worte eines bestimmten Sprechers extrahiert. So lassen sich andere Redner oder Hintergrundgeräusche ausblenden.

Aus zehntausenden Android-Apps lassen sich die Passwörter der Entwickler auslesen

heise online Newsticker - 18. April 2018 - 10:30
Passwörter, PGP-Schlüssel, Krypto-Keys und VPN-Zugangsdaten – in zehntausenden von Android-Apps finden sich Geheimnisse der Entwickler, die eigentlich nicht öffentlich zugänglich sein sollten.

Facebook in der Krise – können Verlage davon profitieren?

heise online Newsticker - 18. April 2018 - 10:00
Gruner + Jahr-Chefin Julia Jäkel wirft Facebook vor, das Sammeln privater Daten zum Geschäftszweck zu machen. Auch andere Medienhäuser sehen das soziale Netzwerk kritischer als früher.

Ersatzteile vom Graumarkt: Apple verliert Klage gegen iPhone-Reparaturshop

heise online Newsticker - 18. April 2018 - 10:00
Apple hat den Bezug unautorisierter, mit Apple-Logo versehener Austausch-Displays als Markenrechtsverletzung erachtet und ist gegen einen unabhängigen norwegischen Reparaturshop vor Gericht gezogen – letzterer bekam in erster Instanz Recht.

PUBG: Entwickler ziehen nach scharfer Kritik Zonen-Änderungen zurück

heise online Newsticker - 18. April 2018 - 10:00
Die tödliche, die Spielkarte verkleinernde Zone ist ein wichtiger Bestandteil eines Battle-Royale-Spiels. Die PUBG-Entwickler hatten ihre Zonen-Timings geändert, was zu starkem Widerspruch in der Community führte. Nun rudern sie zurück.

Börse Stuttgart kündigt App für Kryptogeldhandel an

heise online Newsticker - 18. April 2018 - 10:00
Die erste "Krypto-App, hinter der eine traditionelle Wertpapierbörse steht“ verspricht Sowa Labs, eine Tochter der Börse Stuttgart. Ab September soll man mit der Bison genannten App Kryptogeld handeln können.