Treasure Chest 2: Scalability vs End user convenience.

In reference to OpenGeek’s post, What do you think is the best practice for the next version of Treasure Chest.

The following would be standard in either scenario

• Modular checkout system, easily extendable via subclass of Checkout.php so any programmer can write a paypal, google checkout, etc... checkout.
• Support for multiple options (color, size, flavor, whatever) and support for multiple options on the same item in the cart (a green shirt AND a red, medium shirt (shirt is a single product).
• Support for multiple shipping methods and ability to set shipping calculation by total weight, per item, or flat rate.

As Jason said, there is a lot of overhead if you have many, many products and are using the document interface (and TVs) for setting product attributes, BUT... there is a huge convenience for end users that you may turn the site over to in that they can keep products organized in the document tree, use Ditto to display by tags or document hierarchy, and have the included search functionality.

Putting the products in a separate database table (or tables) will definitely lower the overhead with big catalogs, and will abstract all the product management to an easy module interface, but may result in complex snippet calls and would probably add query strings to the URL which might be something you are trying to avoid.

As many of you know, I am a huge fan of the jQuery javascript library. If a separate table and a module are chosen I am considering Flexigrid for jQuery for the catalog list (see example 3). This would allow searching by product name, category, price, etc...

The original TreasureChest was a closed development environment and, I think, as a result turned out to be of limited use. I want Treasure Chest 2 to be a community project in every aspect, from planning to implementation, to best practices, to documentation (especially documentation!). I think we can all agree that a community driven project will be, by necessity, more useful, extendable, and customizable because there are only so many potential end use situations that a single developer can anticipate, and many eyes make all bugs shallow. Also, if any community can pull this off, it is the MODx community.

I am voraciously awaiting your feedback, and please note that this poll and these comments are not restricted to developers. Anyone who would like to contribute to this project in any way is welcome. I have set up an SVN with google code (for the time being), and will humbly assume the role of primary developer on this project, but many cooks are invited to this kitchen.

I have begun work (rework, started from scratch a dozen times, whatever) on some of the classes for abstracting things like checkouts, a singleton to manage the cart session, and a few other things, but before any hard core coding gets underway, I would like the community to establish some best practice guidelines, and together we can get a general layout of how this should work.

-sD-
Dr. Scotty Delicious, DFPA.

Hi Scotty

what about a combination of the two?  Store the product data in a separate table and manage the product crud with a module. Use TVs to represent custom fields and assign to templates to and create one or more views of the product data. Join the relevant modx tables with your product data table in the module so those fields can be managed.

Use the modx document tree to construct multiple categories, and create a few documents to represent product views (not products themselves).

Your module would need to provide for assignment of products to one or more categories, so you could use a cross reference table to map products > modx documents (which would be the categories).

Similarly, product options could be represented as MODx documents, and presented in your module for attachment to products/categories.
Options
-- Colour
----Red
----Blue
----Green
-- Size
----etc
(You could have a variety of ’option’ templates to represent different types of option, for example a Colour option might have an image tv).

Write an extender for Ditto so you can do product listings with that separate table instead of the site_content table.

Write your own optimised sql for sorting/listing of products, as Ditto would struggle with many thousands of products.

Going on to MODx 0.97, I believe that you would be able to create your own custom resource type instead of a module, and use Ext for all that table sorting goodness.

Could this work?

I like the MODx document approach for smaller shops because it is simple and easy to work with. But for large shops an independent tool would rock. Sounds like a great idea.

Hi Scotty,

about datagrid, have you heard about jqGrid => http://www.trirand.com/jqgrid/jqgrid.html
It offers many nice features such as inline editing.

:-)

Quote from: therebechips at Jun 27, 2008, 09:16 AM

Hi Scotty

what about a combination of the two? Store the product data in a separate table and manage the product crud with a module. Use TVs to represent custom fields and assign to templates to and create one or more views of the product data. Join the relevant modx tables with your product data table in the module so those fields can be managed.

Use the modx document tree to construct multiple categories, and create a few documents to represent product views (not products themselves).
[...]

Write your own optimised sql for sorting/listing of products, as Ditto would struggle with many thousands of products.

Going on to MODx 0.97, I believe that you would be able to create your own custom resource type instead of a module, and use Ext for all that table sorting goodness.

Definitely great ideas. I think built in product search sql is a must. I know a lot of work is going into 0.9.7, but I won’t be focusing on that until a beta is out.
Quote from: dev_cw at Jun 27, 2008, 01:57 PM

I like the MODx document approach for smaller shops because it is simple and easy to work with. But for large shops an independent tool would rock. Sounds like a great idea.

Noted.

Thanks Dev C-dub.

Quote from: heliotrope at Jun 27, 2008, 02:09 PM

Hi Scotty,

about datagrid, have you heard about jqGrid => http://www.trirand.com/jqgrid/jqgrid.html
It offers many nice features such as inline editing.

The style is arguably uglier, but the inline editing is very nice.

-sD-
Dr. Scotty Delicious, DFPA.

For an overly-simplified analogy: do the makers of cash registers make different registers for groceries stores than they do for gas stations, movie theaters or theme parks? Not really. So then I would ask, why do the product catalogs have to be forced into the custom tables or content/tvs mould all? In my mind, how one chooses to implement a catalog is a totally different thing than how one chooses to implement a payment or shipping system (in an ideal world of course). This is what’s so beautiful about FoxyCart because it doesn’t force you down that path like almost every other e-commerce system out there.

Payment processing typically only needs to know a tiny subset of what’s in the cart/who’s making the purchase:
[*] the order total
[*] who’s paying
[*] how they’re paying
(As an aside and not many are doing this now, I think that being able to do authorizations and then actually charge cards on shipping would be a very good thing as well ... and fall in line with the true rules from Visa/MC.)


Similarly, the shipping only needs a few tidbits:
[*] who/where it’s going
[*] how fast it’s supposed to get there
[*] qty/size/weight of packages


One thing that has to be done to be compliant with the new PCI standards for companies taking credit card payments (PCI 6.6) is to either have a web application firewall, or to have code audited professionally. I wonder if the code being open source and reviewed under public scrutiny by a variety of developers qualifies as professional code review and satisfies that requirement?

Ryan,
There is more to a cart system then just the checkout. More information than just the total, who, and how is needed.

A cart needs to be able to display it’s contents (from the session), and Ideally allow the user to make modifications or remove items before they even get to the things you mentioned.
This system, which is certainly a prerequisite for "checkout" means that product details have to be established so that the attributes can be stored in the session, or at least queried for by session identifier when a request is sent to view the cart before payment.

So to address your overly-simplified scenario... The makers of cash registers do not have to make different registers for different stores because all the register does is accept money and make change. The analogy doesn’t really hold water when planning a system to manage a shopping cart to save products to be purchased as the shopper moves between aisles, a scanner to look up the price and options of the products, AND a register to get paid.

What is not absolutely necessary is a cleanly defined way to CRUD products if a universal method can be established for defining products, attributes, and addition/removal of items in the session. However, if a universal method can be agreed on, then a product CRUD system should be easy to implement as well as a handy tool when handing over a store to the end user.

-sD-
Dr. Scotty Delicious, DFPA.

Cart and catalog are likewise two different animals. To be able to generalize something to the degree that it universally accepts all scenarios, attributes, etc. will most likely lead down the path of being overkill and/or over-generalized far too frequently, like trying to come up with a vehicle to win a Forumula One race one week, then turn around and take World Cup (sailing) the next. It’s sorta the same philosophy behind MODx itself. We include a few things that are pretty universal (and some that should probably be removed) for most sites. It’s why we don’t include things like Gallery snippets or Flash players in the base distribution (or insert many other flavors of code to handle different types of content).

To me, a cart would ideally be just a container that holds things to be paid-for and shipped after picking them from the catalog. Carts might even just contain quantities and pointers to a permanent or temporary array of cart_content_IDs. Those cart_content_IDs could exist either in the session or in a DB table, and might themselves contain an array of attributes, etc. (If in the DB, this would enable other things like wish lists, repeat shopping lists, etc. but that’s another topic, for a later discussion.) Maybe the cart line items would have a link to modify the cart_content_IDs ... just thinking out loud and again probably wrong and over-simplifying.

The cart contents should probably then be passed to an order total piece that then determines whether or not qty discounts, coupons, etc. would apply. You definitely should be able to modify the cart contents for sure (quantity or remove, anyway), and ideally once an item is picked in the warehouse during fulfillment and/or shipped, deduct those items from inventory.

At the end of the day, I’d hate to see this turn into another osCommerce or ZenCart that while they’re great projects are trying to be all things to all people. Great discussion!

Great points Ryan.

I think it gets even more complicated when you start to consider inventory management in a scenario where your product is, for example, a t-shirt.
You have sizes in small, medium, and large, and colors in green, blue, and red. Obviously the inventory available for small, red shirts is different from large green shirts, but even more complicated is that small red, medium red, and large red are also going to differ.

I agree that this is a great discussion. Hopefully we can come to some conclusions about where efforts should be focused for MODx e-business solutions. I am, at this point, leaning in the same direction you are hinting at. An API focused cart/checkout with the catalog layout (perhaps using MODx documents, perhaps using a custom interface, maybe something all together different) left up to the individual developer/store owner.

-sD-
Dr. Scotty Delicious, DFPA.

The inventory beast indeed ... add on warehouse bin locations and it gets even more and more complex. My gut says that ultimately, loosely coupled modules will be the way to go, though I don’t know how to couple those yet. There may in fact be different modules for different use cases ... e.g., a shipping/inventory module for straightforward widgets and one for more complex scenarios, like T-shirts in different sizes and colors.

Thoughts on inventory tracking: in the real world for the T-shirt example, I’d probably have a different SKU assigned to each version of the shirts (e.g., red Small = 1, red Medium = 2, green Large = 3, etc.), but I might also have a master "catalog product" variable (and maybe a subcategory variable too) attached to each T-shirt product in my catalog. A GROUP BY on the catalog product/subcategoy and then build the catalog views as needed ... but on the back end you’re still just removing items from inventory as they go out the door based on individual SKUs for each item. You could configure your inventory tracking bits to either point to a custom table or to use a doc/tv combination.

The catalog presentation can be super-simple or crazily-complex (think custom computer builder) ... so I’ll always advocate for avoid the catalog problem in general if at all possible!

Where’s Zap in this conversation?