The Future Of Users, And Impact On Integration

As this is a discussion to return to post 0.9.5, I’ve moved replies to here.

Quote from: OpenGeek at Aug 24, 2006, 09:32 PM

Quote from: PaulGregory at Aug 24, 2006, 07:40 PM

In other news, is there any chance of merging web_users and web_user_attributes for 0.9.5?

I don’t think that’s a good idea just yet. The structure of user identity, IMO, should be kept isolated into three sections, which should be represented by individual classes/tables. These are sections are, credentials/authentication, profile/attributes, and access/permissions, and we already have a close approximation of that model with our current structure (despite the manager/web user chasm). However, I do think some of the credentials-related columns now in user_attributes (a.k.a. the profile class) could possibly be moved to the appropriate user or user_settings tables; things like blocked at least, though I really think each individual column deserves some discussion before we decide where to move anything, since that will likely affect all kinds of legacy deployments, and prevent many core snippets from working without some potentially non-trivial revisions forced on would-be upgraders.

I also want to keep this division moving forward because I have a plan to allow the MODx user framework, to be able to serve as a central source of identity which can aggregate data for each of the three classes of data I mentioned, from whatever repository it is possible to get that data from. Identity data that other applications can then interact with in whatever imaginable way they need to (PHP via MODx API, direct to database from some other language, an Ajax service that returns SAML, or a JSON equivalent, a SOAP service, etc.). This approach of aggregation avoids the need for bi-directional synchronization, and can even allow centralized management of user data if the foreign repository can be written to from MODx. And either the local application continues to have control over it’s standard user system and cares nothing about the MODx aggregation going on, or a bridge can be implemented from the application to take advantage of it.

Fair enough, if it isn’t simple to change the API (I naively assumed that it was), then it does largely make more sense to wait and change things at the point that we merge web and manager users, so we have one upgrade facility. And yes, some serious discussion of the future of users needs to start post 0.9.5.

Your reply basically translates as "either MODx manages it and is the centre, or the other application totally does its own thing with users, because the level of faff is insane."

I fail to see many non-MODx applications that will talk to MODx in such fancy ways without a lot of recoding. Last night I had a go at letting Vanilla use the MODx user table, which was seriously impeded by short-sightedness on the part of Vanilla and the largely pointless use of 2 tables in MODx. A user can only have one set of attributes. A user has to at least have the email attribute, and so that’s 2 tables minimum for a user. I just don’t see how the class division needs to be replicated in the tables. If items were in one easily remappable table, much more data can be shared between apps

Quote from: OpenGeek at Aug 24, 2006, 09:32 PM

...Identity data that other applications can then interact with in whatever imaginable way they need to...

Except the way in which they currently interact with their own data.

If we are to integrate with existing apps, it will be because of common ground in MySQL.

If we are to get other apps to work with the MODx API, then in many cases it is a hack/rewrite. The more you rewrite of a third-party app, the more work you get to do every time they release an upgrade.

I’m concerned. I know that a big user table is inelegant, but you have to consider the speed of integration.

though I really think each individual column deserves some discussion before we decide where to move anything,

Indeed, the fields themselves could do with a rethink.

The way I see it, if you want a split, the logic is:
Required fields in _users, and anything that reasonably might be expected to be useful to other applications or that you wouldn’t expect anyone to have to enter twice.
Optional extra MODx-oriented attributes in _attributes.
Optional extra MODx-oriented settings in _settings.

Which may well only mean moving ’email’ to _users.

I have a plan to allow the MODx user framework, to be able to serve as a central source of identity which can aggregate data for each of the three classes of data I mentioned, from whatever repository it is possible to get that data from.

Great. I take this to mean that we could define where identity data is coming from. But presumably if it is possible to get data for a class from any repository, there is no need to split the tables _users and _user_attributes?

Quote from: PaulGregory at Aug 25, 2006, 03:39 AM

Fair enough, if it isn’t simple to change the API (I naively assumed that it was), then it does largely make more sense to wait and change things at the point that we merge web and manager users, so we have one upgrade facility. And yes, some serious discussion of the future of users needs to start post 0.9.5.

When we have a centralized, and object-oriented API, without code redundancy for everything that goes on with the core tables, we can easily change the API and deal strictly with code migration issues.

Quote from: PaulGregory at Aug 25, 2006, 03:39 AM

Your reply basically translates as "either MODx manages it and is the centre, or the other application totally does its own thing with users, because the level of faff is insane."

Faff? This idea of federated (as in aggregating) user management and identity services is hardly what I would call faff, if by faff you mean "potter about uselessly, be indecisive," and the model I am trying to describe in less technical jargon that I am used to has been implemented before, very successfully, in the J2EE world, where 5 years ago, the state of Java web applications was the same as the majority of PHP web applications, as you describe below. Now, most Java web applications know to either use the J2EE container for identity management, or to use a standards-based web-service to authenticate and get identity data.

Quote from: PaulGregory at Aug 25, 2006, 03:39 AM

I fail to see many non-MODx applications that will talk to MODx in such fancy ways without a lot of recoding. Last night I had a go at letting Vanilla use the MODx user table, which was seriously impeded by short-sightedness on the part of Vanilla and the largely pointless use of 2 tables in MODx. A user can only have one set of attributes. A user has to at least have the email attribute, and so that’s 2 tables minimum for a user. I just don’t see how the class division needs to be replicated in the tables. If items were in one easily remappable table, much more data can be shared between apps

Why do you see the isolation of that data as pointless. I fail to see how it is pointless. And users can have many different uses in a web application, potentially have different profiles from different applications that are involved in the aggregation, etc. And the need for authentication to be a quick and simple process that does not require additional information (beyond the username and password, a.k.a. credentials), definitely justifies the division to me. It’s arguable whether email address should be there as well, but I think in most cases, it is auxilary, and only needed for communication of changes to credentials (resetting password, etc.).

And I do not subscribe to a method of remapping fields in one application to match another. Integration is not that simple in most cases, and this would be the same thing as building a bridge in my model, only the MODx API could be utilized to help easily get the required data for the external application from the aggregation in MODx, even if it is managed from the external application. So each external application serves as a source of aggregation, and each application can, depending on your needs, be bridged back to the identity service, or not. MODx and other apps can still get access to the data exclusively defined for the external app at any time, but unless a specific app needs access to the aggregation, a bridge isn’t necessary. This model also allows for ultimate flexibility in integrating disparate user systems.

Quote from: PaulGregory at Aug 25, 2006, 03:39 AM

Quote from: OpenGeek at Aug 24, 2006, 09:32 PM

...Identity data that other applications can then interact with in whatever imaginable way they need to...

Except the way in which they currently interact with their own data.

If we are to integrate with existing apps, it will be because of common ground in MySQL.

If we are to get other apps to work with the MODx API, then in many cases it is a hack/rewrite. The more you rewrite of a third-party app, the more work you get to do every time they release an upgrade.

I’m concerned. I know that a big user table is inelegant, but you have to consider the speed of integration.

though I really think each individual column deserves some discussion before we decide where to move anything,

Indeed, the fields themselves could do with a rethink.

The way I see it, if you want a split, the logic is:
Required fields in _users, and anything that reasonably might be expected to be useful to other applications or that you wouldn’t expect anyone to have to enter twice.
Optional extra MODx-oriented attributes in _attributes.
Optional extra MODx-oriented settings in _settings.

Which may well only mean moving ’email’ to _users.

I have a plan to allow the MODx user framework, to be able to serve as a central source of identity which can aggregate data for each of the three classes of data I mentioned, from whatever repository it is possible to get that data from.

Great. I take this to mean that we could define where identity data is coming from. But presumably if it is possible to get data for a class from any repository, there is no need to split the tables _users and _user_attributes?

No, the split will be the same; I think this is an important organizational requirement of the data model for user identity and security. And forget MySQL, what about LDAP, Active Directory, SAML from SOAP-based identity servers, Oracle, MS SQL Server, PostgreSQL, etc. -- user data can be found in infinite varieties, and we need to be able to aggregate and integrate as deeply as possible with each one. The model I am trying to describe makes that possible with a minimum amount of effort and maximum amount of flexibility. If an app’s too difficult to hack, let the app alone and have it’s data used in the aggregation of data for the rest of your identity services, but it can continue on without knowledge or concern about the aggregation, as I mentioned.

Aggregating the profile data means additional genericism of the profile/attributes stuff, so caches of the aggregated data can be stored locally, and extended past the base set of MODx attributes with no manual intervention.

Faff was a last-minute substituion for "fannying around required", and I meant it in the "large amount of additional work imposed upon someone in order to achieve what would otherwise be completed more quickly" sense.

Quote from: OpenGeek at Aug 25, 2006, 12:25 PM

Why do you see the isolation of that data as pointless. I fail to see how it is pointless.

Whereas I just don’t see what the point is. Oh, a point is made, but I don’t see anything being achieved. Admittedly, this is partly due to the current implementation of 0.9.x requiring all users to have an email address, which means they *have* to have data in two tables. I just don’t understand why a JOIN is used with two native tables to connect data about one user with more data about that user. I just don’t see how the class division needs to be replicated in the actual tables. Sure, sure, users may have "different profiles from different applications". They will be in different tables. But why should MODx core user data be in two tables? Why? Has this been profiled in some way? Is the number of times that a user logs in and doesn’t need to touch other profile fields such as email or roleID so much greater than the number of times that a user’s profile details are needed together?

If it’s for security, why isn’t there a users table and a user_passwords table? I just fundamentally don’t get it.

Quote from: OpenGeek at Aug 25, 2006, 12:25 PM

If an app’s too difficult to hack, let the app alone

My entire point is that MODx creates the difficulty by sitting there being bloody awkward with its isolated data. I happen to agree long term in rewriting stuff in MODx, but there are many times that a short-term fix is better.

I argue that primary email address is as important as username and password. Of all the data that two systems hold about people, the one piece of data that is most likely to connect them together is email address. The Internet’s only commonly used personal identifier is email address. It is the only logical common field when combining two EXISTING, live web applications. As passwords can be reset via email, email is as important security-wise as username and password. Many systems only let one email address be registered as a user. Many popular systems don’t bother with a username, and just have email and password.

I don’t understand how any aggregation could occur without a primary email address being at the core. I am scared by your comment that email is "only needed for communication of changes to credentials (resetting password, etc.)."

Now, I love the idea of aggregating user info. But it sounds like you’ve thrown out the bi-directional stuff that would make it useful. Does this basically mean that an profile external to MODx is read only? As Admin, do I have to change a banned user’s the access privileges in all the related applications, if all the apps are still using their own stuff? It sounds like the aggregator is just a giant cache.