Logged in: Logged in as: Log in
Register
Locations of visitors to this page SourceForge.net
products how to buy forum bug tracker translate deki company blog
Search:
Page last modified 20:22, 23 May 2008 by petee
User:PeteE > Caching Notes

Caching Notes

              

  • Identify all places to cache
    • GET:pages/{pageid} returns date.modified, PHP needs to use that and set Last-Modified header
      • petee - verified this can be easily accomplished and pushes requests/second to ~ 1800req/s using varnish (1000req/s using squid)
      • petee - Relying on Last-Modified isn't feasible for the following reasons:
        • We don't always updated page.touched: 0004072: page_touched not changed when page metadata is updated
        • Ajax requests (like editing comments) doesn't do a full page reload so the cache doesn't see a new Last-Modified header and mark the cached entry as stale
        • Making changes via the API won't be reflected until the cache expires or an authenticated user views the modified page
    • GET:files/{fileid} sends ETag
      • To avoid a hit to apache (and a 304 if the ETag is found) we should consider using Last-Modified as well
        • we are actually sending Last-Modified so no change is required here
    • feeds
      • GET:site/feed, GET:pages/{id}/feed, GET:users/{id}/feed  - should all be cached in the API
      • Special:RecentChanges - should also send the proper cache headers
        • we probably don't need to explicity purge this but send a much smaller maxage
  • Identify where we need to purge the cache
    • Pages
      • page modified
      • attachment added
      • comment added
    • Attachment added/deleted
      • For our first pass we can just pass through calls to /@api/files/{id} and let the API return the 304 based on ETag
    • both squid and varnish support purging via a PURGE request with the URL as an argument
      • varnish supports regex based purging but I'm not sure how well this will work for Deki Wiki since some pages have Uri's like:  index.php?title=some/page.  We'll most likely have to send individual purge reqests for each child page.  If purching all child pages proves difficult we may have to just set a shorter TTL and allow the breadcrumbs/navigation to be out of sync until the cached version expires
  • We need to disable caching for authenticated users
    • Using varnish, we can easily look for that authtoken= in the cookie and not cache if present
    • TODO: figure out if we can do this in squid using the cookie or if we'll need to move the authtoken in to a header instead       

Max's (awesome) perf numbers

Requests/sec output cache get:pages/{id} output cache get:pages/{id}/contents eAccelerator cache roles,users,services               Notes
8 no no yes yes              

api: ~23%

mysql: ~40%
9.3 yes no yes yes              

api: ~20%

mysql: ~25%
10.3 no yes yes yes              

api: ~15%

mysql: ~20%
13 yes yes yes yes              

api: ~11%

mysql: ~7%
11  yes yes yes no                

api: ~15%

mysql: ~14%
5.5 yes yes no yes            

api: ~5%

mysql: ~4%
4.1 no no no no           

api: ~12%

mysql: ~23%

             

PeteE's perf numbers

Requests/sec output cache get:pages/{id} output cache get:pages/{id}/contents eAccelerator cache roles,users,services

       

 Notes
       no no yes yes                     
       yes no no yes              

api:

mysql:
       no yes yes yes              

api:

mysql:
3.87 yes yes yes yes             

api: 16%

httpd: 25%

mysql: 0%
1.86 yes yes no yes              

api: 7%

httpd: 25%

mysql: 0%
1.79 yes yes no no            

api: 8%

httpd: 25%

mysql: 1%
1.39 no no no no           

api: ~16%

httpd: 25%

mysql: ~2%

     

PeteE's perf numbers (worker MPM, mod_fcgid)       

Requests/sec output cache get:pages/{id} output cache get:pages/{id}/contents eAccelerator cache roles,users,services     varnish,article

cache patch    

      		 Notes
       no no yes yes                     
       yes no no yes              

api:

mysql:
       no yes yes yes              

api:

mysql:
2833.84 yes yes yes yes yes N/A
5.01 yes yes yes yes              

api: 18%

httpd: 26%

mysql: 1%
2.65 yes yes no yes              

api:

httpd:

mysql:
      yes yes no no            

api:

httpd:

mysql:
      no no no no           

api:

httpd:

mysql:

             

     

Tag page
What links here

Files 0

Attach file or image
 

Images 0

 
You must login to post a comment.
Powered by MindTouch Deki v.8.08.2