Database abstraction, total OO refactoring, and more!!!

Quote from: xwisdom at Dec 09, 2005, 12:03 PM

1) Complexity - What is requires to develope code using Propel? I see where jason has pointed that there will be docs to help users setup phing, propel and creaole. This concerns me very much. How easy will it be to develope and package apps in MODx should we go the propel road? Is this a point an click experience? If not then that would dent the dream of a Visual Experience? Another thing comes to mind is access levels. Would a developer need special access to his/her server to run these third party tools?

Patience, I will have a demo as soon as possible for everyone to review. And this will not affect end users at all, other than making it even simpler for them to write snippets that interact with the core (without having to write one-off SQL statements that might break on the next upgrade). Developing and packaging apps with the core, regardless if it’s implemented as is or with the object oriented Propel core is not being changed. We can certainly look at providing ways for users to generate user-defined objects, either via a manager tool or on the fly (e.g. using reflection and visually editable XML definitions or similar), but that is not the purpose of this exercise.

Quote from: xwisdom at Dec 09, 2005, 12:03 PM

2) Performace - Propel is NOT a database abstraction framework, right? It’s an OR mapping framework. Propel uses Creole for DB abstraction, right? My question then what are we pursuing? DB abstraction or OO database?
Propel’s OO layer will most likely be used inside the core APIs which IMO adds processing overhead to these APIs. The performance will not be the same as if we were using straight creole/dbapi or standard SQL syntax, correct?

You are correct, Propel is responsible for the OO mapping and uses (currently) Creole for the DB access layer. And I am going after both OO and DB abstraction. And no one yet knows if the overhead is more, but so far, as I’ve said, indications are good that the framework I’m generating will be just as fast, if not faster than the current implementation. There is no reason to think that the performance will significantly degraded because we are using an OO model on top of a data access layer. In some cases, depending on how the original code was implemented, it will actually preserve resources and be equally as fast.

Quote from: xwisdom at Dec 09, 2005, 12:03 PM

3) Scalability - How well does propel scale when we start talking about hundreds of tables? Sure MODx might not use so many tables but what if someone was to use MODx to create such an application? How well will such an app function under MODx? How many request/second will such a website be able to handle?

We are doing nothing but improving scalability at this point. By introducing transaction support, the Memento pattern for keeping cached objects that are unchanged (and persisting only the actual changes to specific attributes of an object, rather than an entire row of data, as occurs often now), and other tried and tested abstraction patterns, we are making it very easy to build enterprise-class applications against real databases, as well as preparing for the migration to PHP 5 as the rest of the world leaves 4.x behind. And with Creole, you still have access to write optimized SQL queries against the databases; you could craft the largest, richest application in the world, with every feature of Oracle utilized to it’s fullest, or you could choose a custom set of objects using our object model if your prefer (generated or hand-coded). The possibilities and opportunities we will be creating by adopting this architecture are simply undeniable in my view.

Quote from: xwisdom at Dec 09, 2005, 12:03 PM

The Visual Dream is to enable developers to use the MODx interface to do everything from building and packaging web sites and web apps to deploying such packages. Developers should never have to use the command prompt ever.

Again, this all to help make collaborative development of this open source software a real possibility. With the current codebase, it is relatively impossible to refactor the code easily, or even manage which code you need to change or might affect when you do change something. It will reduce redundancy, and reveal all sorts of new possibilities that WILL help provide us with a path to realize this end-user web site nirvana where they can build anything by pointing and clicking and filling out a few form fields.

Okay, quick question (I havn’t read everything so my apoligies if this has already been said).

I agree that a good solid API will make developing much simpler. At this point any kind of use of the API pretty much required poking around in the core code and the database. I think a good solid extensive API will fix that but only if it is well documented. Otherwise people will still have to poke around the core code and I’ll bet that figuring out the gerenated propel code is a lot more difficult than our existing code.

So, is there any sort of documentation generation that can be done with Propel? Or do we need to look at another documentation generation solution or something different? Maybe the propel API documentation will be most of what we need already, I’m not sure but I think it something that will be very important if we want to sell this to snippet/plugin/module developers. It’ll probably make the difference between them loving it or hating it.

We can easily add a task to Phing to generate API docs using whatever code documentation tool we decide to use. Propel itself uses a similar process, so I have a complete project reference with build scripts to illustrate the approach.

And I’m sorry, but our current API makes no use of standard naming conventions or any other pattern that will make usage as simple and consistent as the classes I’ve generated are. What can be simpler than $document->getDocumentsRelatedByParent()? Many-to-many relationships are little more complex, but even then, it’s the difference between calling a single function and writing 20 lines of code to process a complex SQL statement with multiple joins.

Sounds good. And I think the standard naming is absolutely necessary. Consistency is what is needed in every area of the core. It still drives me crazy trying to remember the names of all of the different get, getInfo, getActive etc funtions and each one has different arguments in a different order. A good consistent API with good documentation makes coding exciting and productive instead of slow and frustrating.

I’ll throw in one more comment and say that all of my snippet/plugin/module writing tell me that this is the right way to go. It’s rediculous how much duplication has to go on in the third-party code. The Manager Login snippet is huge, it’s almost a complete mirror of the login processor. It should be:



Well, not exactly, but you get the idea. I really do think thet we need the API and getting it soon and easy with propel sounds good to me. I’ll withold 100% approval until I get my grubby hands on it, but it sounds great.

Don’t get me wrong. Propel is great but these concerns comes to mind when I started look at how Propel works and the number of classes/interfaces that it creates for a single table.

Jason,

It’s good to hear that you’re working on the special modification of Propel that would make thing much easier for us to adopt it.

Jason,

One quick question....

Can Propel be make to go from it’s .xml schema to a .sql on the fly?

My reason for asking this is that it would be cool to use propel’s xml schema to create a database structure then during an install the user selects the database type and the installer (with some embedded propel engine) would then then convert the .xml schema into the appropriate SQL syntax.

Quote from: xwisdom at Dec 09, 2005, 02:22 PM

Can Propel be make to go from it’s .xml schema to a .sql on the fly?

My reason for asking this is that it would be cool to use propel’s xml schema to create a database structure then during an install the user selects the database type and the installer (with some embedded propel engine) would then then convert the .xml schema into the appropriate SQL syntax.

We could make it do it on the fly, but I believe it would be better to have the build generate all the SQL files for our supported target DB’s from the XML schema at build time. This would require a lot less resources to be moved to the server in order to convert the schema at installer run-time.

Quote from: OpenGeek at Dec 09, 2005, 04:03 PM

We could make it do it on the fly, but I believe it would be better to have the build generate all the SQL files for our supported target DB’s from the XML schema at build time. This would require a lot less resources to be moved to the server in order to convert the schema at installer run-time.

Wonderful! I guess we can reserve such explorations for later.