performance.rst 8.3 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224
  1. Performance
  2. ###########
  3. Make your site run as smoothly and responsively as possible.
  4. .. contents:: Contents
  5. :local:
  6. :depth: 2
  7. Can Elgg scale to X million users?
  8. ==================================
  9. People often ask whether Elgg can scale to large installations.
  10. First, we might stop and ask, "where are you planning to get all those users?"
  11. Seriously, though, this is a really interesting problem.
  12. Making Elgg scale is, if anything, an issue of technical engineering.
  13. It's interesting but more or less a solved problem.
  14. Computer science doesn't work differently for Elgg than for Google, for example.
  15. Getting millions of users? That's like the Holy Grail of the entire tech industry.
  16. Second, as with most things in life, the answer is "it depends":
  17. * How active are your users?
  18. * What hardware is Elgg running on?
  19. * Are your plugins behaving well?
  20. `Improving the efficiency of the Elgg engine`__ is an ongoing project,
  21. although there are limits to the amount that any script can do.
  22. __ https://github.com/elgg/elgg/issues?labels=performance&state=open
  23. If you are serious about scalability you will probably want to look at a number of things yourself.
  24. Measure first
  25. =============
  26. There is no point in throwing resources at a problem if you don't know:
  27. * what the problem is
  28. * what resources the problem needs
  29. * where those resources are needed
  30. Invest in some kind of profiling to tell you where your bottleneck is,
  31. especially if you're considering throwing significant money at a problem.
  32. Tune MySQL
  33. ==========
  34. Elgg makes extensive use of the back end database, making many trips on each pageload.
  35. This is perfectly normal and a well configured database server will be able to cope with thousands of requests per second.
  36. Here are some configuration tips that might help:
  37. * Make sure that MySQL is configured to use an appropriate my.cnf for the size of your website.
  38. * Increase the amount of memory available to PHP and MySQL
  39. (you will have to increase the amount of memory available to the php process in any case)
  40. Enable caching
  41. ==============
  42. Generally, if a program is slow, that is because it is repeatedly performing an expensive computation or operation.
  43. Caching allows the system to avoid doing that work over and over again
  44. by using memory to store the results so that you can skip all the work on subsequent requests.
  45. Below we discuss several generally-available caching solutions relevant to Elgg.
  46. Simplecache
  47. -----------
  48. By default, views are cached in the Elgg data directory for a given period of time.
  49. This removes the need for a view to be regenerated on every page load.
  50. This can be disabled by setting ``$CONFIG->simplecache_enabled = false;``
  51. For best performance, make sure this value is set to ``true``.
  52. This does lead to artifacts during development if you are editing themes in your plugin
  53. as the cached version will be used in preference to the one provided by your plugin.
  54. The simple cache can be disabled via the administration menu.
  55. It is recommended that you do this on your development platform if you are writing Elgg plugins.
  56. This cache is automatically flushed when a plugin is enabled, disabled or reordered,
  57. or when upgrade.php is executed.
  58. System cache
  59. ------------
  60. The location of views are cached so that they do not have to be
  61. discovered (profiling indicated that page load took a non-linear amount
  62. of time the more plugins were enabled due to view discovery).
  63. Elgg also caches information like the language mapping and class map.
  64. This can be disabled by setting ``$CONFIG->system_cache_enabled = false;``
  65. For best performance, make sure this value is set to ``true``.
  66. This is currently stored in files in your dataroot (although later
  67. versions of Elgg may use memcache). As with the simple cache it is
  68. flushed when a plugin is enabled, disabled or reordered, or when
  69. upgrade.php is executed.
  70. The system cache can be disabled via the administration menu, and it is
  71. recommended that you do this on your development platform if you are
  72. writing Elgg plugins.
  73. Database query cache
  74. --------------------
  75. For the lifetime of a given page's execution, a cache of all ``SELECT`` queries is kept.
  76. This means that for a given page load a given select query will only ever go out to the database once,
  77. even if it is executed multiple times. Any write to the database will flush this cache,
  78. so it is advised that on complicated pages you postpone database writes until
  79. the end of the page or use the ``execute_delayed_*`` functionality.
  80. This cache will be automatically cleared at the end of a page load.
  81. You may experience memory problems if you use the Elgg framework as a library in a PHP CLI script.
  82. This can be disabled by setting ``$CONFIG->db_disable_query_cache = true;``
  83. Etags and Expires headers
  84. -------------------------
  85. These technologies tell your users' browsers to cache static assets (CSS, JS, images) locally.
  86. Having these enabled greatly reduces server load and improves user-perceived performance.
  87. Use the `Firefox yslow plugin`__ or Chrome DevTools Audits
  88. to confirm which technologies are currently running on your site.
  89. If the static assets aren't being cached:
  90. * Verify that you have these extensions installed and enabled on your host
  91. * Update your .htaccess file, if you are upgrading from a previous version of Elgg
  92. * Enable Simplecache_, which turns select views into browser-cacheable assets
  93. __ https://addons.mozilla.org/en-us/firefox/addon/yslow/
  94. Memcache
  95. --------
  96. Memcache is a generic caching technology developed by Brad Fitzpatrick for LiveJournal.
  97. .. warning:: SUPPORT FOR MEMCACHE IS EXPERIMENTAL AND MAY BE CHANGED.
  98. Installation requirements:
  99. | `` * php5-memcache``
  100. | `` * memcached``
  101. Configuration:
  102. Uncomment and populate the following sections in settings.php
  103. .. code:: php
  104. $CONFIG->memcache = true;
  105. $CONFIG->memcache_servers = array (
  106. array('server1', 11211),
  107. array('server2', 11211)
  108. );
  109. Squid
  110. -----
  111. We have had good results by using `Squid`_ to cache images for us.
  112. .. _Squid: http://en.wikipedia.org/wiki/Squid_cache
  113. Bytecode caching
  114. ----------------
  115. There are numerous PHP code caches available on the market.
  116. These speed up your site by caching the compiled byte code from your
  117. script meaning that your server doesn't have to compile the PHP code
  118. each time it is executed.
  119. Hosting
  120. =======
  121. Don't expect to run a site catering for millions of users on a cheap shared host.
  122. You will need to have your own host hardware and access over the configuration,
  123. as well as lots of bandwidth and memory available.
  124. Memory, CPU and bandwidth
  125. -------------------------
  126. Due to the nature of caching, all caching solutions will require memory.
  127. It is a fairly cheap return to throw memory and CPU at the problem.
  128. On advanced hardware it is likely that bandwidth is going to be your bottleneck before the server itself.
  129. Ensure that your host can support the load you are suggesting.
  130. Configuration
  131. -------------
  132. Lastly, take a look at your configuration as there are a few gotchas that can catch people.
  133. For example, out of the box, Apache can handle quite a high load.
  134. However, most distros of Linux come with mysql configured for small sites.
  135. This can result in Apache processes getting stalled waiting to talk to one very overloaded MySQL process.
  136. Check for poorly-behaved plugins
  137. ================================
  138. Plugins can be programmed in a very naive way and this can cause your whole site to feel slow.
  139. Try disabling some plugins to see if that noticeably improves performance.
  140. Once you've found a likely offender, go to the original plugin author and report your findings.
  141. Use client-rendered HTML
  142. ========================
  143. We've found that at a certain point, much of the time spent on the server
  144. is simply building the HTML of the page with Elgg's views system.
  145. It's very difficult to cache the output of templates since they can generally take arbitrary inputs.
  146. Instead of trying to cache the HTML output of certain pages or views,
  147. the suggestion is to switch to an HTML-based templating system so that the user's browser can cache the templates themselves.
  148. Then have the user's computer do the work of generating the output by applying JSON data to those templates.
  149. This can be very effective, but has the downside of being significant extra development cost.
  150. The Elgg team is looking to integrate this strategy into Elgg directly,
  151. since it is so effective especially on pages with repeated or hidden content.