Suggested hardware and hosting requirements for large Elgg sites.

Overview
As the popularity of Elgg grows, we’re getting more and more questions regarding the optimal hosting environment for large networks. We’ve created this post in response to that. It contains a growing, consolidated list of suggested hosting and hardware requirements for running large (5k+ user) Elgg installations. It’s based on community contributed best practices, Elgg’s documentation and Arck’s experience with the platform over the past two years. It’s intended to be used as a point of reference and makes no claim of being all-inclusive. If you have a suggestion or edit, please include it in a comment and we’ll review it for inclusion in the next version.

Many thanks to the Elgg community for all of the input contributed across Elgg forums and mailing lists.

OS / Software Requirements

• Dedicated LAMP (Linux, Apache, MySQL 5+, PHP 5.2+) based server
• Apache mod_rewrite module and multibyte string support
• GD for graphics processing
• JSON (included in PHP 5.2+)
• memcached (100-150MB of RAM allocated)

Hardware Requirements

• CPU: Quad 3ghz+
• RAM: 4GB+
• HDD: 1-2TB+ / SSD (Disk space can be estimated by multiplying the amount of space you wish to allocate to each user by the total number of users. This could range between 10MB and 100MB depending on your requirements.)

Optimizations

MySQL

• Convert the following MySQL tables to the InnoDB engine: elggannotation, elggentities, elggentity_relationships, elggmetadata, elggmetastrings, elggusers_sessions.
• Leave all remaining tables as MyISAM for FULLTEXT searching
• Raise the InnoDB buffer size to utilize roughly 50% of your total RAM.
• Add OPTIMIZE queries to a nightly cron for all Elgg tables

PHP

• Increase the total amount of memory allocated to PHP from 8MB in PHP’s configuration file. We recommend at least 16MB.
• Change the maximum uploaded file size in PHP’s configuration to at least 2MB.

Scripts

• Minify all JS and CSS
• Serve JS and CSS from separate sub-domains (i.e. css.yourhost.com and js.yourhost.com)

Plugin Execution Order

• Plugins are loaded in the order they appear on the plugin administration page. The theme plugin should be loaded near the bottom of the list, as it applies styles to other plugins.

Load Testing
The hu_skawa_genusers Elgg plugin can be used to simulate any number of concurrent users on an Elgg site. We recommend volume testing using this plugin in combination with http_load, or any other load testing tool.

Load Balancing
For large Elgg sites, it is critical to isolate the database(s) from the web server. In a load-balanced configuration (see typical example below) the web servers are horizontally scalable. The load balancing technology could be an appliance from a known vendor like Cisco, Big IP, etc or simply a Linux host implementing the Linux-HA project.

At the database layer the simplest solution is an active passive approach with enough hardware horsepower to handle the expected query load. Load and volume testing is required in order to determine that is needed in your own environment. Additionally, a true active/active database cluster could be implemented at a higher cost using a commercial clustering solution from MySQL.

elgg load balanced configuration

Arck launches public Elgg-based social network demo: Kyoutsuu

Kyoutsuu is a basic example of what’s possible with Elgg, an open-source social networking framework, and Arck Interactive, a team of talented web designers developers. Elgg provides the building blocks that allow you to create and grow your own social networking site, whether publicly (like Facebook) or internally on a networked intranet (like Microsoft Sharepoint).

Arck provides a full-range of custom design and development services for Elgg and offer options for either hosting your social network with us, or on your own server. You can view a fill list of our Elgg services and case studies for social networks we’ve developed with Elgg on our web site.

Please register for a free account and play in our sandbox. If you have any questions, or feedback, click the little Feedback tab on the left side of the web page and we’ll get back to you soon!

By the way, kyoutsuu means community or commonality in Japanese.

What’s possible with a free, open-source social networking framework called Elgg? The Wiley Faculty Network, for one.

We’ve been using the open-source social framework Elgg on nearly all of our social development project over the past two years. Never heard of Elgg? I’m not surprised. It hasn’t been given the attention or the visibility that it absolutely deserves. Hands down, it’s the best framework for developing custom social networks.

That said, I’d like to share our latest Elgg development effort, a faculty network for John Wiley & Sons, that we developed in cooperation with Garfield Group Interactive.

The Wiley Faculty Network is a global community of faculty, connected by a passion for teaching and a drive to learn, share, and collaborate. Members connect to collaborate with colleagues, join professional and peer groups, find mentors, attend virtual and live events, and view a wealth of resources all designed to help them grow as educators.

Site functionality included: one-click login integration with Wiley’s existing faculty system ProfVal; role-based access with custom profiles; intelligent dashboard widgets that suggest content to users based on their professional and personal interests; a custom content management system plugin allowing Wiley to deliver context and discipline specific content to users; a complete events management system; custom file/resource (documents, videos, audio) sharing; granular analytics and reporting; blogs, user groups and forums.

You can read the complete case-study on the Arck Interactive site, here:

• http://www.arckinteractive.com/work/wiley-elgg-development/index.php.

The live Wiley Faculty Network web site is available here:

• http://wfn.wiley.com/home

Elgg Development Services Page Launched

We’ve launched a page dedicated to the Elgg development services we provide. Since we started offering Elgg services nearly 2 years ago, the demand for custom development has steadily increased. It’s now at the point where we’re spending about half our time developing new social networks with Elgg.

Check out the page here: http://www.arckinteractive.com/services/elgg-development.php.

6 free tools for web developers

As a web developer and small business owner, I’m on a constant quest to find free tools that allow greater collaboration and communication for my project teams, which tend to be spread throughout the country.

Without further adieu, here are my top picks:

1. Etherpad
Etherpad is a real-time, web-based collaborative text editor. You create a “pad”, invite others to join by sharing the URL, and paste in, or type, text. Edits show up in real time with each user’s text formatted in a different color. There’s a chat room where you can discuss your edits and an option to save revisions.

I’ve found Etherpad especially useful for team editing CSS and Javascript files and sharing bits of code for debugging. The only thing it’slacking is syntax highlighting for more languages, but I hear that’s in the works.

Interesting note: Etherpad was created by ex-Google Docs developers with funding from y-combinator.

2. Pastebin
Pastebin is a collaborative debugging tool with syntax highlight for just about every programming language. You paste in a code snippet, select the syntax and it outputs a web page with your formatted code including line numbers. It’s useful for sharing small bits of code on mailing lists, or IRC.

3. Snipplr
Snipplr is a social source code repository for scripts, CSS and code snippets. Forgot the min-height fix for IE6? They’ve got it. Need a javascript/CSS rounded corner script? There are several.

4. Xdebug
Xdebug is an essential PHP extension that provides stack/function tracing, time tracking, parameter display, code coverage analysis and a performance profiler that allows you to find bottleneck methods. If you can’t parse cachegrind files, this guy has released a PHP-based parsing tool.

5. Teamviewer
Teamviewer is a browser based remote desktop sharing application and a great alternative to paid applications like Goto meeting. It has supports remote desktop control/sharing, file transfers. The one feature the free version is lacking is audio, but that’s easily solved with Skype or FreeConference (see below)

6. Freeconference
Freeconference is a tool for setting up conference calls. The free versions allows you to set up a conference line with a local (not toll free) exchange. Participants dial in to the number and enter an access code to join your conference.