|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Tuesday 17 February 2009 8:24:15 pm
Hi,
I've tested the extension on a varnish installation nearly the same .vcl file you provided as example.
Usually we work with Custom Headers like:
[HTTPHeaderSettings]
CustomHeader=enabled
Pragma[/]=
Cache-Control[/]=public, must-revalidate, max-age=10800
Expires[/]=10800
So Varnish is able to cache the request to the eZ publish backend.
Using your extension works great to login into the backend through varnish.
When the user is logged in, the requests are passed to the backend.
But there's a problem when the user has already visited some pages on the site.
The browser has a local cache for the requests and uses them instead of requesting once again.
How do you force the browser to request once again ?
Working without custom headers would effect that varnish wouldn't cache the most load intensive requests (eZ Publish backend).
Any ideas?
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Wednesday 18 February 2009 8:56:00 am
Hi Norman,
first of all I have to apologize because I uploaded the wrong .vcl configuration files in the doc/ folder of the extension. I removed it in trunk and will be replaced by a .vcl for Varnish1 and another for Varnish2, but I need time to do this and unfortunately I am too busy at the moment :/
So Varnish is able to cache the request to the eZ publish backend.
Using your extension works great to login into the backend through varnish.
When the user is logged in, the requests are passed to the backend.
What does your vcl_recv function look like ?
Something like this worked fine on http://haz.de for example :
[...]
sub vcl_recv
{
[... lots of backend switch stuff here ...]
/* do not use Varnish when the user is authenticated */
if(req.http.Cookie ~ "^is_logged_in=true.*$")
{
pass;
}
if (req.request == "GET")
{
lookup;
}
if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico)$")
{
lookup;
}
}
[...]
But there's a problem when the user has already visited some pages on the site.
The browser has a local cache for the requests and uses them instead of requesting once again.
I agree I already got the same issue, unfortunately I did not find a solution for this one yet.
I generally reduce the TTL to a few minutes, from 1 to 5, and this is usually acceptable by customers but this is only a workaround and not a solution though.
How do you force the browser to request once again ?
Currently, we can not.
What we need here is a correct Etag support in eZ Publish.
If think that modifying the Etag's value would solve your issue here, however this is actually not possible, but maybe in 4.1, with the new content_read trigger, you could write a custom workflow event which generates a new Etag value for the current node or the current object.
Since the object has changed, its modification date changed as well. Then taking this date into account and using it to generate a different Etag should force your browser to revalidate its copy with the server. However you will have to remove your Expires call because if I remember well Expires will avoid HTTP request, but if you have both Cache-Control and Expires, then Cache-Control will override Expires and the browser will test if its local copy is still fresh or not.
Hope that helps.
(I hope my answer is not too messy)
:)
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Wednesday 18 February 2009 6:54:51 pm
Hi Jerome,
I've tested a bit around with some settings and found a solution that seem to work fine in most cases.
[HTTPHeaderSettings]
CustomHeader=enabled
HeaderList[]=Cache-Control
HeaderList[]=Pragma
Cache-Control[/]=public, must-revalidate, max-age=10800
Pragma[/]=
with the vcl:
sub vcl_recv {
[... backend & default stuff ...]
# do not use Varnish when the user is authenticated
if (req.http.Cookie && req.http.Cookie ~ "is_logged_in" ) {
pass;
}
if (req.request != "GET" && req.request != "HEAD") {
pipe;
}
if (req.http.Expect) {
pipe;
}
if (req.http.Authenticate) {
pass;
}
if (req.http.Cache-Control ~ "no-cache") {
pass;
}
# Cache all other objects which use cookies (overwrite default).
# throw away cookies all other and lookup
remove req.http.cookie;
lookup;
}
sub vcl_fetch {
[... default stuff ...]
if (obj.http.Set-Cookie && (!obj.http.Set-Cookie == "is_logged_in")) {
remove obj.http.Set-Cookie;
}
[... default stuff ...]
deliver;
}
As long as the user has no "is_logged_in" Cookie, he'll get the cached version of the site.
As soon as the logs in, the requests are passed to the backend server.
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Thursday 19 February 2009 9:17:52 am
Hello Norman,
I wrote a very similar .vcl file yesterday evening, however it does not seem to work for me.
Here is what I got so far.
As you will notice there are a few debugging stuff but the concept is there anyway
sub vcl_recv
{
set req.backend = default;
/***************************************************/
/* Never cache pages with POST parameters */
/* Typical use case : /user/login or /usr/register */
/* You do not want to get those pages cached */
/***************************************************/
if (req.request == "POST")
{
pipe;
}
/******************************************************/
/* do not use Varnish when the user is authenticated */
/* This configuration requires the ezvlogin extension */
/* http://projects.ez.no/ezvlogin */
/* With the default configuration for LoginSettings */
/* in vlogin.ini.append.php */
/******************************************************/
//if(req.http.Cookie ~ "^is_logged_in=true.*$")
if (req.http.Set-Cookie ~ "^is_logged_in=true.*$")
{
pass;
}
/********************************************************/
/* Returns the item from the cache for any GET request */
/* All the configuration done in HTTPHeaderSettings in */
/* eZ Publish as well as Apache's mod_expires will come */
/* into play here */
/********************************************************/
if (req.request == "GET")
{
lookup;
}
/*********************************************************/
/* We want Varnish to return binary files from its cache */
/* all the configuration we used for mod_expires is used */
/* in this case */
/*********************************************************/
if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico)$")
{
lookup;
}
}
sub vcl_fetch
{
/**
* Might work with eZ Publish 4.0.1 and with and without eZ Vlogin
*/
//if (obj.http.Cookie ~ "^eZSESSID=.*$")
/**
* Works with eZ Publish trunk only
*/
// WORKS !!!
if (obj.http.Set-Cookie ~ "^is_logged_in=deleted.*$")
{
set obj.http.Is-Logged-In="deleted";
// DOES NOT WORK !!
//remove obj.http.Cookie;
// WORKS !!!
remove obj.http.Set-Cookie;
}
// WORKS !!!!
else if (obj.http.Set-Cookie ~ "^is_logged_in=true.*$")
{
set obj.http.Is-Logged-In="true";
}
set obj.http.X-Cache = "Miss";
}
sub vcl_hit
{
if (!obj.cacheable)
{
pass;
}
set obj.http.X-Cache = "Hit";
deliver;
}
I will send the correct .vcl file as soon as I get everything working fine.
I do not know your feeling about Varnish but even though it is really powerful, it is also very easy to make mistakes without noticing it, and testing can be really difficult.
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 9:46:09 am
This is what I currently have:
sub vcl_recv {
# normalize encoding
if (req.http.Accept-Encoding){
if (req.http.Accept-Encoding ~ "gzip"){
set req.http.Accept-Encoding = "gzip";
}
elseif (req.http.Accept-Encoding ~ "deflate" ) {
set req.http.Accept-Encoding = "deflate";
}
else{
remove req.http.Accept-Encoding;
}
}
if (req.url ~ "\.(js|css|jpg|png|gif|mp3|swf|flv|xml|html|ico)$"){
remove req.http.cookie;
}
# pass the request if Cookie from vlogin is present
if (req.http.Cookie && req.http.Cookie ~ "is_logged_in" ) {
pass;
}
if (req.request != "GET" && req.request != "HEAD") {
pipe;
}
if (req.http.Expect) {
pipe;
}
if (req.http.Authenticate) {
pass;
}
if (req.http.Cache-Control ~ "no-cache") {
pass;
}
# Cache all other objects which use cookies (overwrite default).
# throw away all other cookies and lookup
remove req.http.cookie;
lookup;
}
sub vcl_hit {
if (!obj.cacheable) {
pass;
}
deliver;
}
sub vcl_fetch {
if (!obj.cacheable) {
pass;
}
# remove cookie information if the user isn't logged in
if (obj.http.Set-Cookie && (!obj.http.Set-Cookie == "is_logged_in")) {
remove obj.http.Set-Cookie;
}
if ( obj.http.Pragma ~ "no-cache"
|| obj.http.Cache-Control ~ "no-cache"
|| obj.http.Cache-Control ~ "private") {
pass;
}
if (obj.ttl < 3600s){
set obj.ttl = 3600s;
}
deliver;
}
Mainly it throws away any cookie information.
Additionaly I'm normalizing the encoding for all browsers.
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Thursday 19 February 2009 10:24:13 am
You were right for the remove req.http.cookie; at the end of vcl_recv, it seems to work fine.
I finally came up with the following configuration, which is a bit different than yours but it seems to work. We (eZ) should really provide "official" Varnish configuration files....
backend default
{
set backend.host = "127.0.0.1" ;
set backend.port = "8080" ;
}
sub vcl_recv
{
set req.backend = default;
if (req.request == "POST")
{
pipe;
}
if(req.http.Cookie ~ "^is_logged_in=true.*$")
{
pass;
}
if (req.request == "GET")
{
lookup;
}
if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico)$")
{
lookup;
}
/* Nirvana */
remove req.http.cookie;
}
sub vcl_fetch
{
/**
* Should Works with eZ Publish 3.*, 4.0.* with eZ Vlogin
*/
//if (obj.http.Set-Cookie ~ "^eZSESSID=.*$")
/**
* Works with eZ Publish trunk only
*/
if (obj.http.Set-Cookie ~ "^is_logged_in=deleted.*$")
{
remove obj.http.Set-Cookie;
}
}
:)
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 10:49:27 am
The .vcl seems fine. I'll give it a try later today and see what happens.
I agree with you about the standard which should be supported by eZ,
also the cookie "is_logged_in" cookie should be standard.
Additionally we need to have a cookie like "has_user_data".
For example when a anonymous user adds something to the basket or to the whishlist.
This could easily done by adding a few lines to the modules.
Currently I hacked the basket.php like:
// Set eZVLogin cookies
$user = eZUser::instance();
eZVLoginHelper::setUserCookie( $user, $user->attribute('contentobject')->attribute('name') );
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Thursday 19 February 2009 11:15:31 am
Actually I still got a problem with the issue described in your first post. My test was a false positive :/. I tried with your configuration file and mine, same problem in both cases.
But I am thinking about a potential workaround for this.
To sum up the issue is the following :
- You are anonymous, you get the page from Varnish or your local cache.
- You log in and are redirected to the previously visited page and tadahh you get your local copy, which is of course not what you want, you want a fresh page, like if there is no-cache, no-store no-xxxx.
The idea is the following :
Since, in ezvlogin we control the way users are authenticated, how about adding a specific marker to Varnish to say "Hey I just loged in send no-cache|store|whatever because I want
a fresh copy, and not my local one".
This could be a cookie or a specific HTTP header. I would not like adding yet another cookie because we already have too much of them on the eZ Publish' plate... But I would prefer sending a specific HTTP header that Varnish could understand, something like :
Just-Logged-in=yes, and if Varnish gets this header it returns no-cachexxxx and remove this header after that of course to keep things clean and clear.
What do you think about this ?
I will write a patch for ezvlogin anyway and see what happens, if it work then I will commit it to trunk.
[...]also the cookie "is_logged_in" cookie should be standard.
As of 4.1 you wil get such a cookie, is_logged_in=true when logged in and is_logged_in=delete for anonymous users.
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 11:25:20 am
How are your headers looking like?
Currently I use:
[HTTPHeaderSettings]
CustomHeader=enabled
HeaderList[]=Cache-Control
HeaderList[]=Pragma
Cache-Control[/]=public, must-revalidate, max-age=10800
Pragma[/]=
which works fine.
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Thursday 19 February 2009 12:17:54 pm
Ha ok you use must-revalidate, not me.
Will add right away.
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 12:29:50 pm
Another thing I modified is the behavior of ETags within Apache.
I removed them all...
Header unset ETag
FileETag None
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Thursday 19 February 2009 12:30:34 pm
Does not work either with your .vcl and mine, I tried with the following headers :
Cache-Control[]
Cache-Control[/]=public, must-revalidate, max-age=200
Pragma[]
Pragma[/]=
Expires[]
Expires[/]=+200
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 12:36:13 pm
Try removing the Expires header
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Thursday 19 February 2009 12:44:37 pm
This should not change things as Cache-Control should override Expires when both are set :
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.21
Note: if a response includes a Cache-Control field with the max-
age directive (see section 14.9.3), that directive overrides the
Expires field.
I will test without Expires anyway, we never know ;)
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 12:48:37 pm
|
|
|
Jérôme Renard
Member since:
06 August 2008
Posts: 22
|
Thursday 19 February 2009 12:58:19 pm
Well it seem that removing the Expires field helped.
This is weird though, good to know anyway :)
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 12:59:18 pm
if (obj.http.Set-Cookie ~ "^eZSESSID=.*$")
should be
if (obj.http.Set-Cookie ~ "^eZSESSID.*=.*$")
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 1:07:18 pm
Should we talk about skype ?
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Thursday 19 February 2009 1:20:13 pm
I've tested a bit around and the .vcl seems to be fine now,
when working without the expires header:
sub vcl_recv
{
if (req.request == "POST")
{
pipe;
}
if(req.http.Cookie ~ "^is_logged_in=true.*$")
{
pass;
}
if (req.request == "GET")
{
lookup;
}
if (req.url ~ "\.(gif|jpg|jpeg|swf|css|js|flv|mp3|mp4|pdf|ico)$")
{
lookup;
}
/* Nirvana */
remove req.http.cookie;
}
sub vcl_fetch
{
/**
* Works with eZ Publish 3.*, 4.0.* with eZ Vlogin
*/
if (obj.http.Set-Cookie ~ "^eZSESSID.*=.*$")
{
set obj.http.X-Cookie = "Cookie removed";
remove obj.http.Set-Cookie;
}
}
sub vcl_hit
{
if (!obj.cacheable)
{
pass;
}
set obj.http.X-Cache = "Cache-Hit";
deliver;
}
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Friday 20 February 2009 9:18:37 am
Hi Jerome,
I found out that
if (req.request == "POST")
{
pipe;
}
causes some trouble, as soon as the client sends a post, he'll enter the pipe mode which means that alls requests are passed to the backend as long as the connection is available.
This happens for exaple when using ajax functions like "update viewcount".
|
|
|
Norman Leutner
Member since:
09 January 2008
Posts: 44
|
Friday 20 February 2009 9:21:50 am
Removing that part works fine.
|
|
