My last thoughts on Symfony2

Well I’ve been playing with Symfony2 for 3 months now and today is the day I’m officially turning my back on the Symfony project. I played around and built several small things just to see how it worked. I found it to have a brutal learning curve but maybe thats just me. Ultimately performance is what has driven my decision. I’ve heard the arguments “early optimization is one of the 7 deadly sins of programming” and “hardware is cheap”. All I have to say to both of those arguments is bollocks!

First off:

“early optimization is one of the 7 deadly sins of programming”
Sure, I can understand that – write your app, then optimize it. But your underlying framework should be absolutely optimized. If you start with a slow base guess what the outcome is going to be… a slow website/application. From what I’ve read online, Symfony2 is probably a huge performance increase over Symfony however when you strip Symfony2 down to its core framework without Doctrine or anything else and simple do:

 public function someAction() { return new Response('Hello World'); } 

I only got 950 requests per second, before adding on any baggage at all. I made a simple site, the homepage has 2 queries and thats it. 160 requests per second. Seriously? Someone posted on my other Symfony post that “you could pick another framework that is 20% faster….”, so lets look at some other frameworks for a second:

Out of the box, (H)MVC “Hello World” tests:

DooPHP: 8800 requests per second
FuelPHP (5.3 framework): 5000 requests per second
Codeigniter 2: 4800 requests per second
Kohana: 4000 requests per second
Yii: 3900 requests per second
Cake: 1300 requests per second

All tests were done on exactly the same server, running exactly the same software stack, and all taking network out of the equation. When you added network into the equation, my server was able to deliver about 700 requests per second of a full content index page of a website, still about 5 times more than the same page built in Symfony2 using 2 doctrine queries. The test machine is a 768MB of RAM, quad processor, virtual machine from Linode.

“Hardware is cheap”
I’ve heard this argument plenty of times. Sure the hardware might be cheap but that doesnt factor in the cost of the people required to make your application run on a multi server configuration or the cost of building and configuring that hardware to work well. Building multi server deployments comes with a whole host of challenges and most people don’t have an app that can scale horizontally at the flip of a switch. Funny that all the worst performing platforms all say “hardware is cheap”. Magento is a great example of this philosophy. Quite possibly the best marketing scam ever, Magento is a joke when it comes to performance. But I digress…

Symfony2 is slow, very very slow. Regardless of caching, the core framework is more than 5 times slower than FuelPHP (another 5.3 framework). If that is the sacrifice you are willing to make then good on you. Supposing you wanted to service hundreds of thousands of requests per day. With a performance driven framework you could do it with a single dedicated box which for arguments sake costs you $200 per month. With Symfony you’d be looking at $1000 per month. Thats an awful lot of additional tshirts, coffee mugs, or iphone apps that you’d have to sell each month just to pay your hosting (let alone the more specialised staff for multi server setups).

Symfony2, can I please get a credit for the last 3 months of my life. Thanks.

30 Replies to “My last thoughts on Symfony2”

  1. Very interesting point of view. I what did you use to run the performance tests? I have been using Symfony for just a bit longer than you, and I haven’t noticed performance issues on my local environment – however I’ve never tested formally or taken notice of how well it handles on production.

  2. Hi, I’m very interested in this topic, could you be more precise on how did you run your tests? How did you manage to count the requests? What commands did you run? I’m learning Symfony2 at the moments, and i’m forced to agree, if symfony 1 wasn’t simple, Symfony2 is even harder to master.

  3. Please delete the previous comment. I thought about it. It would be much more interessting to know how you have done this benchmark. Maybe with development enviroment features turned on?

  4. I have the following questions for you:
    – have you bothered to run against the production environment?
    – have you bothered to check this part of the documentation about the performance
    – have you bothered to check what Doctrine2 can offer in terms of query caching and how to use it?
    – since you just jumped into using ORMs, real frameworks, PHP 5.3, are you sure you know what you are doing?

    And now from my experience, I’m using Symfony2 on one of the largest webstore site in Eastern Europe with clients from more that 6 countries, and after reading on how to optimize for performance either the ORM part or the Symfony2 part, we achieved some impressive performance with it, avg. response time of 68 ms, with the highest response time of 126 ms, when we have more that 3000 customers on at our peaks hours on a single server (I can’t provide the exact number of customers nor the website name due to various legal stuff). The config of the server is: 16GB RAM, 8 core processor, and we have the highest load reported of 1.32 with and avg. of 0.5. Also the machine uses Apache 2.something, and we have a nginx server in front of it for caching images/js/css files and act as a load balancer.

  5. Do you use the Hello World Edition for your tests ?
    With the standard edition, there are a lot of things not useful for a hello world test.

  6. Symfony is not a framework to do hello world. Of course, if you’re doing small applications (like hello word), use something else ( in that case, I don’t even see why you’re using a framework, just do a die(‘hello world’); ) . For small project, I would rater go with Lithium, or if really you want to stay with symfony, but optimized for smaller projects, go with Silex.

    Symfony is like a huge tools set, it’s a framework that’s useful for large development. Reading your post, I was thinking: “It’s like saying a pick up is not a fast car. It’s not intended to be fast, nor confortable. Now try to pull 10 tons of rocks in a Lamborgini…”

  7. I agree with the other commenters. I don’t think that “out of the box” tests are meaningful for performance testing this sort of framework. The full Symfony2 stack is doing a whole lot of work. If you’re writing a simple application, you can remove most of the Symfony2 stack or you can use something else. But I don’t believe that saying it is slow is accurate. I’ve only used Symfony2 for a few months, but I have worked with quite a few frameworks over the last few years, and Symfony2 compares favourably with all of them.

    And it’s learning curve is steep, but the whole framework also maintains a level of sanity that I haven’t found in any other framework I’ve used. After three months of fairly intensive use, basically every issue I’ve had with it has come back to user error (mine).

  8. Ok… well this post definitely got a lot more responses than I was expecting and some very valid points. Because this post is couple of weeks old now and I can’t remember all the details off the top of my head, I’m going to setup a test again and give it another shot just to make sure I’m not overlooking something.

    In the meantime, I’ll address some of your comments…

    Ryan: Out of the box were the other framework tests. My tests on Symfony2 involved commenting out lots of stuff to make sure the core framework was all that was loading without Doctrine and all the other bits and pieces that come packaged in the standard install. I definitely took sometime to get Symfony as barebones as I could. I’ve had plenty of the user error problems along the way, Symfony offers a lot of features that are (quite honestly) well outside my knowledge of OOP design patterns.

    Francois: I understand the point you are trying to make however to continue your analogy – wouldn’t you rather have a chasis that is light and strong which you can then bolt a large engine to (a server), add a trailer (ORM, whatever), or if you just want a “lambo”, throw a thin skin over the top and strip away everything that isnt necessary and still get high performance? I absolutely understand that “hello world” is a crap test and that if you wanted real performance you’d just make a html file with “hello world” inside it because as soon as you use PHP you’re going to take a huge performance hit anyway compared to the speed of a static file. I’m actually going to try Silex out as it appears to be exactly what I’m after for testing Symfony2 at its lightest. Thanks.

    Inophage: If by “Hello World” edition you mean the Acme bundle ‘Hello World’ demo then no. I created my own bundle which simply had a single controller with a single method which simply had: return new Response('Hello World'); I just did a quick google and haven’t found any obvious links to a special edition.

    Charlie: No, I was using app.php, definitely not app_dev.php.

    Dlsinper: All my tests were on my production server, as were all the other frameworks I tested. As I mentioned earlier in this reply – I can’t remember the finer details however I’ll apply that page to my next test (which should be tomorrow sometime). With regard to ORM – I’ve had little experience with it however I do have MySQL configured with query caching and ample amounts of RAM available to it for something as trivial as a couple of basic queries to output some article stubs. With regard to my knowledge of all of these things – I’m not 100% sure that I’m not making an error and I will publish this next trial so you can all critique it. Thank you for sharing your server stats, thats interesting, can I ask what you’d roughly see on a daily/monthly basis for traffic and how much of that RAM is used for your caching strategy or is most of it for Apaches PHP threads? I personally take Apache completely out of the equation and just use Nginx + PHP with PHP-FPM enabled. I also use APC for opcode caching.

    Josiah, Tito, and Lennart: When I do my next lot of testing (hopefully tomorrow depending on how my day shapes up), I’ll document exactly what the setup is, how I have Symfony2 configured and how I’m testing the performance. Typically I use a combination of ApacheBench and Siege depending on what I’m trying to achieve. Siege I usually setup up on a bunch of VPS’s and then try and bring my test server to its knees.

    Celso: I’m curious too, how can Dailymotion handle it? Given that I know nothing about the hosting infrastructure that drives daily motion. I’d take a wild stab in the dark and say that large servers, copious amounts of caching, CDN, and plenty of other optimisation tricks to reduce the work the server(s) has to do.

    As a parting note – this article is purely my opinion and based off my experience(s) alone. I don’t claim to be a Symfony2 guru nor do I have a Zend cert or any other formal programming qualification. I do however build Nginx based hosting platforms and I do have a good knowledge of squeezing performance out of the hardware and I do know what sort of resource utilization I see on the server when I’m testing.

    Thank you everyone who has commented, hopefully in the next few days I can get up another post to either support this one or admit my own error and walk away with my tail between my legs πŸ˜‰

  9. I think your post takes a very purist approach to the whole matter.

    I think it’s completely possible that other frameworks are faster, and I think we’d all love to see those stats. The important thing to remember is how things fare in the real world. In the real world there are other limitations (such as bandwidth pipe) and many factors to consider (features of the framework, documentation, community, runs on latest version of php etc).

    One of the most important factors people often fail to consider is the audience/visitor numbers. It might be true that other frameworks can handle 5x more request per second, but each developer has to ask themselves, how many requests do I think I’ll get, and would I be happy with optimisation tricks like caching & CDN.

    Now onto the interesting stuff, it’s clear you have an audience keen to talk about this topic, and I love the fact that you’re interesting in the performance of the framework you’re choosing to work with. If you could post those benchmarks, such that others could verify/critique them you’ll find two things, 1) you should get some well deserved credibility for your attempts to remain transparent & to provide useful stats to the community, & 2) Others, fanboys, geeks, experts can chime in and give advice on how realistic these figures are, and possibly even give pointers as to how to modify the tests so they match the real world, or provide some notable difference in performance.

    Finally I’d like to mention, although this was only your own personal account of things, one of the visuals I like to use when making decisions is a matrix, for example I like comparing the following, rather then taking such a narrow approach.

    – Speed benchmarks
    – Was the framework designed to run on the latest version of PHP
    – Documentation
    – Community Support
    – Licence Free vs Commercial
    – Learning Curve (Time required to learn Framework specific code)
    – Ease of use (After learning curve)
    – Standout Features of Framework (ie does it auto generate code, does it have an authentication class, does it hook into popular api’s)
    – Code Cleanliness & Stability (This could be a matter of opinion, however some code is spagetti, some code takes a purist approach and has 1 million files ie Magento, other Frameworks are just full of errors)
    – Framework Background (Who makes the Framework, have they been around long, will the be around for a while, are they backed by a company or living on a prayer, do they have a huge talent base through an open source community)

  10. I’ve made a test too with Symfony2 and can confirm you that is slow.
    I’ve tested a blog in each of the frameworks/CMS that were subject in the benchmark: Symfony2, Codeigniter2.1.1, Django 1.4 and Drupal 7.
    For Symfony2 I used:
    For Django we used:
    In Drupal and Codeigniter we made blogs as similar as possible to these projects.

    Symfony2 and Drupal were the slowest. Codeigniter and Django were the fastest.

    However the difference between Symfony2/Drupal 7 and Codeigniter/Django were pretty big. We measured the performance with NewRelic.
    Symfony2/Drupal7 had a maximum of 1500 requests per minute while Codeigniter/Django at 5000 requests per minute.

    If this is not a proper test (without caching like pagecache for Drupal, Varnish for Symfony2) what other tests can you do?

    I understand that you can optimize it further, but if you start with a slow framework it’ll be harder to optimize it in the end if you know you want top notch performance. I’m not thinking about the case were you have many static pages and use Varnish.
    I’m talking about dynamic websites, with Category listings, searches, and new content coming in all the time. Can you use stuff like Varnish to save you then?

    Oh.. and the response times.. Symfony2/Drupal7 over 200ms, Codeigniter and Django at about 10ms!

    By the way, we’re currently using Drupal 7, but we have high expectations, and optimizing it to reach Codeigniter performances (over 4000 rpms with 10ms response times) is hard, if not impossible without adding more hardware!

  11. I’m not sure why one would compare Django (a python framework) to Symfony (a PHP framework) as they are very different animals.

    i’d love to see the tests used to determine speed. And remember the tests should be run not as a barebones framework, but rather with everything turned on like it would be in production. Some things which may make the “base” framework seems slow, increase speed of components built on top of it.

    1. Pretty sure I didn’t mention Django in this post but whatever. Django and Symfony are both MVC frameworks, they might be 2 different languages but they essentially do the same thing. Both have ORMs, both are modular, both have templates (Twig is just a clone of Django templates). I think its totally reasonable to compare the two to get an idea of relative performance between 2 frameworks and languages.

      Python and Django took me a week to build a site the first time I ever touched either technology. Symfony took me months to be able to make a whole site. I’m not talking brochure-ware sites, I’m talking about complex sites hooked up to multiple external APIs, with auth, and lots of forms. The only part of Django I’ve struggled with at all is the Forms API.

      I don’t remember what the tests were any more (I tried to put it out of my mind 6 months or so ago), I spent ages testing and tuning Symfony trying to get a reasonable number of requests per second out of it.

    1. I never said you couldn’t handle large amounts of traffic with a slow framework. What I said was that you’ll need more hardware to power your web app.

      Here are the take away points re:Symfony2 from that article:

      • Choosing a web framework doesn’t mean killing performances as long as you use a solid architecture in front of it.
      • Longer than expected learning curve
      • Staffing

      To be honest I have no real problem with Symfony2, its just not for me – I found it frustrating to learn and I found its performance seriously lacking. Since I made the switch to Python I’ve never looked back at PHP. Currently I’m looking into Pypy and although I’ve built quite a few sites in Django and I’ve played with Tornado as well. I find Python to be infinitely more readable than PHP and I usually get a lot more done with less code with the added bonus of significant performance gains.

  12. I got 10 requests per second with symfony for a simple hello world. But you know what? it does not matter!

    first of all, the time it needs to run a hello world is more or less the same time it needs to execute queries and render a complex page. The time it took for the hello world is not for the echo itself, but to load and parse settings… so it is not fair to test it on that echo.

    Second, it is worth learning, it makes development much faster, it took me 6 month to learn it well, and I hated it. But when it was done I realized it was worth it.

    And as for the performance, you should use varnish cache. Whether you can do 100 requests per second or 500 with raw php, varnish will let you serve thousands per second easy. So you should be caching anyway, why would it matter how much the raw performance is?

  13. A week ago I started to use Zend 2, coming from my codeigniter days. I thought it was horribly over-engineered. Yesterday I started with Sympfony2 and all I can say is how much I love Zend !!!!

  14. If you want to waste time, use Symfony2. Such a giant waste of time for the stupidiest things!!! Your framework is not going to program your application. You must program your application, so learn to code proper PHP, and use simpler frameworks / your own.

    1. As a developer, you must focus on the business logic. The rest is job of the framework. With a simpler framework or you own you will need to put effort in other place. Symfony2 is the framework that most allow you to focus on your business logic in any given situation.

  15. @drtete it’s interesting that you talk about stupid things and then you close your sentence with “use your own”… what could be more stupid than creating/using your own framework? Especially these days where there are so many options to choose from, plus lots of those frameworks have been tested in real world examples.

  16. Basically he’s wanting/suggesting that fmoerwarks should be able to scale out of the box.Provide sharding, hashing users to the same R/W master, db pooling etc etc. All the stuff you generally have to write for yourself when a site needs to scale.Symfony is the same as any other framework, if you can’t do it you have to write it yourself.I liked his graph comparing writing yourself and writing with a framework where the time plummets when you hit against something the framework can’t do .When you are developing you need to make sure that you don’t do this, because if the framework can’t do it not really a problem as the frame work is written in your language of choice it means you can just code your solution to fit into your framework.Sometimes this means you could just write a plugin, for example what we did for Symfony to use XSLT as our templating language.Also some of the issues that can slow things down is confusing the M in MVC to be the objects provided by the ORM.You should model the M s you need if they happen to fit exactly the M s provided by the ORM great job done but sometimes in complex applications that are not just CRUD they don’t. If your in this mindset before you code with your framework of choice you won’t hit against the dip as much.

  17. I struggled with symfony for 6 months before leaving php and switching to python. I tell you it is not the symfony alone it is the whole php thing. I dislike php and anything associated with that language

Leave a Reply

Your email address will not be published. Required fields are marked *