Software is Like an Onion · Thursday, September 6, 2007
Ben Bangert, of the excellent Pylons project, wrote a really nice article describing some of his issues with the Elixir project that I have been working on for a while. I think he makes some excellent points, many of which I totally agree with.
However, one of the core principles of his arguments against using Elixir is that Elixir is a “layer” on top of SQLAlchemy. The argument basically states that any layer adds additional complexity, and increases the amount that you have to know, which is at some level true. However, there isn’t a piece of software in the world which doesn’t introduce layers of abstraction to simplify certain types of problems. Ben’s own Pylons project is a layer on top of the Paste project, which is a layer on top of WSGI. SQLAlchemy has all sorts of layers, from its low-level layer on top of the DB-API, up to its own ORM, which is a layer on top of the lower level database packages.
SQLAlchemy is awesome, and I have a bunch of projects with I have worked on that use plain SQLAlchemy, without any of the ORM. I also have worked on projects that use SQLAlchemy with its standard ORM syntax, and I have worked on projects that opt to go with Elixir. I think all three approaches are valid, depending on what problems you are trying to solve. If Ben’s argument was dead on, then the best approach would always be to write all of your own code, relying on no external libraries or layers of abstraction. I think Ben is really oversimplifying the problem, and not seeing the real issues with Elixir.
So, the long and short of it is that, while I respect everything that Ben has to say (we talk fairly frequently, and he is one sharp cookie, to be sure), I think his “layers” argument is a very silly one. Software has layers of abstraction, and sometimes you find that you bump into those abstractions, and have to jump down to a lower level layer to solve your problems. No big deal, as long as its possible to sidestep your abstractions.
Ben’s post, along with some other posts he links to, does make a few things clear:
- A lot of people find Elixir’s statement syntax for defining fields and relationships to be off-putting.
- Elixir is poorly documented, which makes it difficult to break abstractions, and leads people to think that it isn’t capable of doing things that it can do just fine, like compound primary keys.
A lot of effort is being put in right now by the core Elixir developers to help solve these problems. We’ve seen the light and listened to our users, so the current version of Elixir in subversion supports a traditional Pythonic syntax, closer to ActiveMapper, TurboEntity, and SQLObject:
class Book(Entity):
title = Field(Unicode)
owners = ManyToOne('Person')
class Person(Entity):
name = Field(Unicode)
books = OneToMany('Book')
Elixir also provides you many ways to drop down to lower levels to access your SQLAlchemy table objects directly, but they aren’t well documented. Elixir is a very young project, and has come a long way in a short amount of time, but like every young software project, we could use some help in making Elixir easier to use by improving documentation. We’re constantly listening to feedback from users, and trying to adjust to their needs. We know that Elixir isn’t the right level of abstraction for all problems, but it certainly fits the mold for lots of people out there, and helps them quickly solve a lot of their problems.
Comment
- If there’s any lesson to be learned from TurboGears, it’s that your project can’t afford to spend much time with poor or minimal documentation, lest the entire effort begin to stagnate or lose momentum. I know documentation isn’t fun, but it has to be a priority if there’s going to be long-term success.
— Mike Pirnat 1058 days ago # - By a layer, what I mainly meant was that its one more layer a new user has to go through to try and do things that appear fairly basic in the SQLAlchemy docs.
Consider a new user setting up the Entity’s you show there, and seeing you use Book.query. They go to the SQLAlchemy docs to understand how to do some more query options… however the query there looks rather different. Instead of:
Book.query.filter_by(...)
They see:
Session.query(Book).filter_by(...)
Again, no one who knows Elixir well is going to be too confused, but those just learning this have to do additional mental translation their entire way through the SA docs as they try and see how to apply it to Elixir. This is an additional layer of complexity to wade through (again, a seasoned Elixir user will likely not have any problem with this).
However, on occasion, even with some of my Elixir hacking, I have been utterly dumbfounded how to do somethings that are easy when using just SQLAlchemy. But Elixir puts me through additional steps to pull out table objects and find relations it setup that I forgot how they’re named (what was the name of that m2m table? I forgot, and its not in my module so I have to start looking for docs on what happened).
Elixir has a rather large documentation challenge, because it needs to not only explain what its done, but where its put all the SQLAlchemy options, settings, etc. so that a new user can figure out how to relate their Elixir Entity to the SQLAlchemy docs. As a framework designer, its my belief that Elixir is a choice for an advanced user, not a new user. And as the SQLalchemy docs are so thorough, even advanced users will likely have an easier time maintaining a big project using plain SQLAlchemy.
— Ben Bangert 1058 days ago # - Have you thought about doing more doctests as documentation? I am all fired up on doctests as I just started using them :)
— Noah Gift 1058 days ago # - Thanks for the comments Ben. Yes, I think Elixir has a documentation challenge, just like any other project. In fact, I think it has a challenge very similar to Pylons, which not only has to document its own code, but other people’s code too (WSGI, Paste, SQLAlchemy, Beaker, Routes, etc).
I have started to use Pylons (via TurboGears 2.0) on a regular basis now, and at first, I had lots of problems, because I didn’t understand what Pylons was doing under the hood, with things being placed into my modules namespaces, middleware being set up, paste, beaker, etc. I have written lots of straight WSGI apps, and lots of TurboGears apps, and found Pylons to be confusing.
But, that was just at first! It took me about a week or two to filter through what documentation is available (which is significantly more than Elixir’s right now), ask questions on IRC, dig through the code, and eventually understand what was going on under the covers. Now that I understand, I am much more productive in Pylons than I would be with a straight WSGI app, and I really like it! Its extremely powerful, and I am so glad I put in the time to understand it.
I think Elixir will prove to be the same way for lots of people, and we have a long way to go in terms of being simpler, cleaner, and better documented.
Your point about Elixir not being good for brand-new users is well-taken. I think people would be much better off understanding a bit about SQLAlchemy before they learn Elixir. The slides from my recent talk on Elixir took this approach, and people in the audience immediately grasped what was going on, and lots of those people are now happy users :)
This has been a great discussion!
— Jonathan LaCour 1058 days ago # - Nice articles and points made on both “sides” of the discussion.
As a relative newcomer to Pylons, I’ve dabbled with both SQLAlchemy on its own and with Elixir, and I agree with Jonathan’s points that each approach has its place; in this case, Choice is Good. I could easily imagine opting for one over the other for particular projects, especially as Elixir matures and my familiarity with both APIs improves. And to look at it another way, if faced with the choice I’d rather have SQLA with Elixir than not have SQLA at all, i.e. some other ORM (possibly putting documentation aside for the present).
I do agree with Ben on the subject of Elixir potentially being deceptively more difficult for new users, at least with the current state of the documentation. I think much of that might be attributable to the Active Record pattern itself though, rather than any disparities between Elixir and SQLA. In limited experiences with Rails, what looks so simple quickly turns into hurdles in trying to accomplish what I want with ActiveRecord’s DSL. Not that it can’t be done, just that, as Ben says, it can be an additional layer of abstraction to wade through. Again, I suppose it’s a trade-off to be considered case-by-case, and even Fowler suggests that AR “is a good choice for domain logic that isn’t too complex”.
Ultimately, each project certainly holds demonstrable value for me, and I thankfully look forward to the continued innovations of both.
— Ches Martin 1057 days ago #
commenting closed for this article