Custom system settings namespace list not showing all items

45 Posts

ram Reply #1, 13 years, 3 months ago

Hi all—

I use custom system settings/namespace to hold variables that are rarely edited and are used across all areas of a site.

I was just editing this list on a recently upgraded site (2.2.5 -> 2.2.8) and I noticed that some had disappeared! I had a look in the database and they were there, so I went back and tried to create another one and that disappeared too.

I looked around a bit and noticed that there are actually more listed just they are not being shown and there is no next page button active to show you there are more available to view.

There are 22 in this namespace, with the initial default view of 30 per page, however it lists "21 of 22". If I change the number per page to 100 it shows all of them.

I have very similar numbers (and more) on other sites but this listing issue is not in evidence.

EDIT:

I was just working on another site (my MODX template file, recently updated 2.2.7 > 2.2.8), I added several items to my settings, which brought the total there to 26. It only shows 24 out of 26 now on initial view (30 per page), as with other site setting view to 100 shows all. [ed. note: last edited this post 13 years, 3 months ago.]

24,544 Posts

BobRay Reply #2, 13 years, 3 months ago

Do the missing settings (especially the first one that doesn't show) contain spaces, carriage returns, or non-alphanumeric characters?

Are there any blank entries in the DB table?

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting

45 Posts

ram Reply #3, 13 years, 3 months ago

There doesn't seem to be very much interesting at all about the data there!

I am not sure if my post was clear. The records are there, just that they are hidden beyond a seemingly random number until the display per page number is changed. It counts them correctly, just doesn't display until this number is changed and doesn't provide a next page button to view the additional records. It hides anything added beyond a random number it seems to choose, I think in one case 21, and this case 24. It just doesn't show the last records in the list.

I have found this on 2 different servers so far.

I exported the table and pulled out the bits from the namespace I have been working on, a slightly sanitised and stripped down version of the modx_system_settings table here:

SET SQL_MODE="NO_AUTO_VALUE_ON_ZERO";

CREATE TABLE IF NOT EXISTS `modx_system_settings` (
  `key` varchar(50) NOT NULL DEFAULT '',
  `value` text NOT NULL,
  `xtype` varchar(75) NOT NULL DEFAULT 'textfield',
  `namespace` varchar(40) NOT NULL DEFAULT 'core',
  `area` varchar(255) NOT NULL DEFAULT '',
  `editedon` timestamp NOT NULL DEFAULT '0000-00-00 00:00:00' ON UPDATE CURRENT_TIMESTAMP,
  PRIMARY KEY (`key`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8;

('x_user_tracking_enabled', '0', 'combo-boolean', 'xsystem', 'Search Engine Optimisation', '0000-00-00 00:00:00'),
('x_user_tracking_code', '', 'textfield', 'xsystem', 'Search Engine Optimisation', '0000-00-00 00:00:00'),
('x_css_file_dev', '/assets/css/style-dev.css', 'textfield', 'xsystem', 'Site Style', '2013-03-27 08:01:52'),
('x_css_file_pub', '/assets/css/style.css', 'textfield', 'xsystem', 'Site Style', '2013-03-27 08:01:30'),
('x_js_framework_file', '//ajax.googleapis.com/ajax/libs/jquery/1.9.1/jquery.min.js', 'textfield', 'xsystem', 'Site Scripting', '2013-03-27 07:35:43'),
('x_js_file_dev', '/assets/js/behaviour-dev.js', 'textfield', 'xsystem', 'Site Scripting', '2013-03-27 08:03:34'),
('x_js_file_pub', '/assets/js/behaviour-min.js', 'textfield', 'xsystem', 'Site Scripting', '2013-06-22 12:06:34'),
('x_css_dev_enabled', '0', 'combo-boolean', 'xsystem', 'Site Style', '2013-03-27 08:40:46'),
('x_js_dev_enabled', '0', 'combo-boolean', 'xsystem', 'Site Scripting', '0000-00-00 00:00:00'),
('x_twitter_username', 'example', 'textfield', 'xsystem', 'Social Media', '2013-04-15 07:12:46'),
('x_social_media_img', '', 'textfield', 'xsystem', 'Social Media', '0000-00-00 00:00:00'),
('x_google_plus_page', '', 'textfield', 'xsystem', 'Social Media', '2013-06-20 10:55:38'),
('x_facebook_page', '', 'textfield', 'xsystem', 'Social Media', '0000-00-00 00:00:00'),
('x_linkedin_profile', '', 'textfield', 'xsystem', 'Social Media', '0000-00-00 00:00:00'),
('x_email_addr', '[email protected]', 'textfield', 'xsystem', 'Contact', '0000-00-00 00:00:00'),
('x_phone_no', '', 'textfield', 'xsystem', 'Contact', '0000-00-00 00:00:00'),
('x_mobile_no', '00000 000 000', 'textfield', 'xsystem', 'Contact', '2013-06-22 09:18:15'),
('x_latitude', '', 'textfield', 'xsystem', 'Location', '0000-00-00 00:00:00'),
('x_longitude', '', 'textfield', 'xsystem', 'Location', '0000-00-00 00:00:00'),
('x_addr_building', '', 'textfield', 'xsystem', 'Location', '0000-00-00 00:00:00'),
('x_addr_locality', '', 'textfield', 'xsystem', 'Location', '0000-00-00 00:00:00'),
('x_addr_postcode', '', 'textfield', 'xsystem', 'Location', '0000-00-00 00:00:00'),
('x_addr_street', '', 'textfield', 'xsystem', 'Location', '0000-00-00 00:00:00'),
('x_addr_city', '', 'textfield', 'xsystem', 'Location', '0000-00-00 00:00:00'),
('x_js_modernizr_file', '', 'textfield', 'xsystem', 'Site Scripting', '0000-00-00 00:00:00'),
('x_js_framework_file_fallback', '', 'textfield', 'xsystem', 'Site Scripting', '0000-00-00 00:00:00'),

24,544 Posts

BobRay Reply #4, 13 years, 3 months ago

If you delete all files in the core/cache directory, do they all show up?

How about if you clear the browser cache and cookies?

Looking at the System Settings getlist processor, I don't see what could cause that other than a cache issue (either the MODX cache or your browser's JavaScript cache).

Browser cache issues are common when you upgrade MODX.

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting

45 Posts

ram Reply #5, 13 years, 3 months ago

Cleared browser cache, cleared site cache, flushed all sessions, deleted contents of /core/cache/, deleted all cookies from that domain.

Still there I'm afraid. Shows "Displaying 1 of 24 of 26" by default. I notice if I set to show 31 per page I see 1 extra, 32 per page I can see all.

All system settings I have set are still usable programatically, just don't show without changing view number.

I noticed last night updating add-ons that the package management list doesn't seem to be able to keep consistency, I browser add-ons and the first page may show 10 items, the next 9, the next 8 (not as logical as that, the numbers seem a bit more random, but close to this behaviour).

Very odd!

24,544 Posts

BobRay Reply #6, 13 years, 3 months ago

Very odd indeed. We're getting into pretty weird territory, but is there any JavaScript running that might cause a variable collision?

The next thing, I guess, would be to use Chrome Dev Tools while the page is displaying and, on the Network Tab, check the actual data being sent to the getlist processor in the request (the limit variable should be going across there), and the actual data being returned in the response.

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting

45 Posts

ram Reply #7, 13 years, 3 months ago

Yes, it is odd.

I am a bit lost looking at what reveals itself in the Network monitor!

Am I right understanding relevant info is in settings.php and going to getList? I can see listings there with data regarding the list view (seems to be counting right too in the parts I can see), but there are quite is a lot of activity there so I may have missed something!

From settings.php going to getList—

I note it seems to realise there is a limit of 30 set (see headers). Full number is also listed in the response, but only 25 results received.

Headers:

Request URL:http://x.lektriq.com/connectors/system/settings.php
Request Method:POST
Status Code:200 OK
Request Headersview source
Accept:*/*
Accept-Encoding:gzip,deflate,sdch
Accept-Language:en-US,en;q=0.8
Connection:keep-alive
Content-Length:121
Content-Type:application/x-www-form-urlencoded; charset=UTF-8
Cookie:__utmz=91703559.1366809164.1.1.utmccn=(direct)|utmcsr=(direct)|utmcmd=(none); __utma=91703559.2142382170.1366809164.1367277576.1371917980.4; __utmc=91703559; PHPSESSID=t8n7tk52en3qjsm8n2j537rs12
Host:x.lektriq.com
modAuth:modx51bfaf2d394a89.40282101_151c66b7af23567.37766289
Origin:http://x.lektriq.com
Referer:http://x.lektriq.com/manager/index.php?a=70
User-Agent:Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/27.0.1453.116 Safari/537.36
X-Requested-With:XMLHttpRequest
Form Dataview sourceview URL encoded
start:0
limit:30
action:getList
namespace:xsystem
area:
HTTP_MODAUTH:modx51bfaf2d394a89.40282101_151c66b7af23567.37766289
Response Headersview source
Cache-Control:no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Connection:Keep-Alive
Content-Encoding:gzip
Content-Length:1505
Content-Type:text/html; charset=utf-8
Date:Mon, 24 Jun 2013 15:28:14 GMT
Expires:Thu, 19 Nov 1981 08:52:00 GMT
Keep-Alive:timeout=2, max=100
Pragma:no-cache
Server:Apache
Set-Cookie:PHPSESSID=t8n7tk52en3qjsm8n2j537rs12; path=/; httponly
Vary:User-Agent,Accept-Encoding

Preview (only partially unfolded), full data shown below in response.

results: [{key:x_email_addr, value:[email protected], xtype:textfield, namespace:xsystem, area:Contact,…},…]
0: {key:x_email_addr, value:[email protected], xtype:textfield, namespace:xsystem, area:Contact,…}
1: {key:x_mobile_no, value:00000 000 000, xtype:textfield, namespace:xsystem, area:Contact,…}
2: {key:x_phone_no, value:00000 000 000, xtype:textfield, namespace:xsystem, area:Contact,…}
3: {key:x_skype_name, value:skypename, xtype:textfield, namespace:xsystem, area:Contact,…}
4: {key:x_test_item, value:0, xtype:textfield, namespace:xsystem, area:Contact,…}
5: {key:x_addr_building, value:Building Name, xtype:textfield, namespace:xsystem, area:Location,…}
6: {key:x_addr_city, value:Big City, xtype:textfield, namespace:xsystem, area:Location,…}
7: {key:x_addr_locality, value:Exampletown, xtype:textfield, namespace:xsystem, area:Location,…}
8: {key:x_addr_postcode, value:X66 3AA, xtype:textfield, namespace:xsystem, area:Location,…}
9: {key:x_addr_street, value:99 Example Street, xtype:textfield, namespace:xsystem, area:Location,…}
10: {key:x_latitude, value:0, xtype:textfield, namespace:xsystem, area:Location,…}
11: {key:x_longitude, value:0, xtype:textfield, namespace:xsystem, area:Location,…}
12: {key:x_user_tracking_code, value:UA-00000000, xtype:textfield, namespace:xsystem,…}
13: {key:x_user_tracking_enabled, value:0, xtype:combo-boolean, namespace:xsystem,…}
14: {key:x_js_dev_enabled, value:0, xtype:combo-boolean, namespace:xsystem, area:Site Scripting,…}
15: {key:x_js_file_dev, value:/assets/js/behaviour.dev.js, xtype:textfield, namespace:xsystem,…}
16: {key:x_js_file_pub, value:/assets/js/behaviour.min.js, xtype:textfield, namespace:xsystem,…}
17: {key:x_js_framework_file, value://cdnjs.cloudflare.com/ajax/libs/zepto/1.0/zepto.min.js,…}
18: {key:x_js_framework_file_fallback, value://cdnjs.cloudflare.com/ajax/libs/jquery/1.10.1/jquery.min.js,…}
19: {key:x_js_modernizr_file, value://cdnjs.cloudflare.com/ajax/libs/modernizr/2.6.2/modernizr.min.js,…}
20: {key:x_css_dev_enabled, value:0, xtype:combo-boolean, namespace:xsystem, area:Site Style,…}
21: {key:x_css_file_dev, value:/assets/css/style-dev.css, xtype:textfield, namespace:xsystem,…}
22: {key:x_css_file_pub, value:/assets/css/style.css, xtype:textfield, namespace:xsystem, area:Site Style,…}
23: {key:x_facebook_page, value:https://www.facebook.com/, xtype:textfield, namespace:xsystem,…}
24: {key:x_google_plus_page, value:https://plus.google.com/‎, xtype:textfield, namespace:xsystem,…}
total: "28"

Response:

{"total":"28","results":[{"key":"x_email_addr","value":"[email protected]","xtype":"textfield","namespace":"xsystem","area":"Contact","editedon":"Jun 24, 2013 08:14 AM","name_trans":"Email Address","description_trans":"Public-facing email address","area_text":"Contact","description":"Public-facing email address","name":"Email Address","oldkey":"x_email_addr"},{"key":"x_mobile_no","value":"00000 000 000","xtype":"textfield","namespace":"xsystem","area":"Contact","editedon":"Jun 24, 2013 08:14 AM","name_trans":"Phone Number - Mobile","description_trans":"Public-facing mobile number","area_text":"Contact","description":"Public-facing mobile number","name":"Phone Number - Mobile","oldkey":"x_mobile_no"},{"key":"x_phone_no","value":"00000 000 000","xtype":"textfield","namespace":"xsystem","area":"Contact","editedon":"Jun 24, 2013 08:15 AM","name_trans":"Phone Number - Landline","description_trans":"Landline phone number","area_text":"Contact","description":"Landline phone number","name":"Phone Number - Landline","oldkey":"x_phone_no"},{"key":"x_skype_name","value":"skypename","xtype":"textfield","namespace":"xsystem","area":"Contact","editedon":"Jun 24, 2013 08:14 AM","name_trans":"Skype Name","description_trans":"Skype username","area_text":"Contact","description":"Skype username","name":"Skype Name","oldkey":"x_skype_name"},{"key":"x_test_item","value":"0","xtype":"textfield","namespace":"xsystem","area":"Contact","editedon":"Jun 24, 2013 01:00 AM","name_trans":"Test Item","description_trans":"This is a test item","area_text":"Contact","description":"This is a test item","name":"Test Item","oldkey":"x_test_item"},{"key":"x_addr_building","value":"Building Name","xtype":"textfield","namespace":"xsystem","area":"Location","editedon":"Jun 24, 2013 08:19 AM","name_trans":"Address - Building","description_trans":"Building Name","area_text":"Location","description":"Building Name","name":"Address - Building","oldkey":"x_addr_building"},{"key":"x_addr_city","value":"Big City","xtype":"textfield","namespace":"xsystem","area":"Location","editedon":"Jun 24, 2013 08:18 AM","name_trans":"Address - Town\/City","description_trans":"Town or city","area_text":"Location","description":"Town or city","name":"Address - Town\/City","oldkey":"x_addr_city"},{"key":"x_addr_locality","value":"Exampletown","xtype":"textfield","namespace":"xsystem","area":"Location","editedon":"Jun 24, 2013 08:18 AM","name_trans":"Address - Locality","description_trans":"Local area town\/village","area_text":"Location","description":"Local area town\/village","name":"Address - Locality","oldkey":"x_addr_locality"},{"key":"x_addr_postcode","value":"X66 3AA","xtype":"textfield","namespace":"xsystem","area":"Location","editedon":"Jun 24, 2013 08:17 AM","name_trans":"Address - Postcode","description_trans":"Postcode","area_text":"Location","description":"Postcode","name":"Address - Postcode","oldkey":"x_addr_postcode"},{"key":"x_addr_street","value":"99 Example Street","xtype":"textfield","namespace":"xsystem","area":"Location","editedon":"Jun 24, 2013 08:16 AM","name_trans":"Address - Street","description_trans":"Street address - number and thoroughfare","area_text":"Location","description":"Street address - number and thoroughfare","name":"Address - Street","oldkey":"x_addr_street"},{"key":"x_latitude","value":"0","xtype":"textfield","namespace":"xsystem","area":"Location","editedon":"Jun 24, 2013 08:16 AM","name_trans":"Latitude","description_trans":"Office location","area_text":"Location","description":"Office location","name":"Latitude","oldkey":"x_latitude"},{"key":"x_longitude","value":"0","xtype":"textfield","namespace":"xsystem","area":"Location","editedon":"Jun 24, 2013 08:16 AM","name_trans":"Longitude","description_trans":"Office location","area_text":"Location","description":"Office location","name":"Longitude","oldkey":"x_longitude"},{"key":"x_user_tracking_code","value":"UA-00000000","xtype":"textfield","namespace":"xsystem","area":"Search Engine Optimisation","editedon":"Jun 24, 2013 08:20 AM","name_trans":"User Tracking Code","description_trans":"User tracking code","area_text":"Search Engine Optimisation","description":"User tracking code","name":"User Tracking Code","oldkey":"x_user_tracking_code"},{"key":"x_user_tracking_enabled","value":"0","xtype":"combo-boolean","namespace":"xsystem","area":"Search Engine Optimisation","editedon":"Jun 24, 2013 01:00 AM","name_trans":"User Tracking","description_trans":"Enable\/disable user tracking","area_text":"Search Engine Optimisation","description":"Enable\/disable user tracking","name":"User Tracking","oldkey":"x_user_tracking_enabled"},{"key":"x_js_dev_enabled","value":"0","xtype":"combo-boolean","namespace":"xsystem","area":"Site Scripting","editedon":"Jun 24, 2013 01:00 AM","name_trans":"JS Development Mode","description_trans":"Enable JS development mode to show development JS to admin users.","area_text":"Site Scripting","description":"Enable JS development mode to show development JS to admin users.","name":"JS Development Mode","oldkey":"x_js_dev_enabled"},{"key":"x_js_file_dev","value":"\/assets\/js\/behaviour.dev.js","xtype":"textfield","namespace":"xsystem","area":"Site Scripting","editedon":"Jun 22, 2013 09:42 PM","name_trans":"JS Development File","description_trans":"Location of development JS file.","area_text":"Site Scripting","description":"Location of development JS file.","name":"JS Development File","oldkey":"x_js_file_dev"},{"key":"x_js_file_pub","value":"\/assets\/js\/behaviour.min.js","xtype":"textfield","namespace":"xsystem","area":"Site Scripting","editedon":"Jun 22, 2013 09:43 PM","name_trans":"JS Public File","description_trans":"Location of public JS file.","area_text":"Site Scripting","description":"Location of public JS file.","name":"JS Public File","oldkey":"x_js_file_pub"},{"key":"x_js_framework_file","value":"\/\/cdnjs.cloudflare.com\/ajax\/libs\/zepto\/1.0\/zepto.min.js","xtype":"textfield","namespace":"xsystem","area":"Site Scripting","editedon":"Jun 23, 2013 10:16 AM","name_trans":"JS Framework File","description_trans":"JS framework file location","area_text":"Site Scripting","description":"JS framework file location","name":"JS Framework File","oldkey":"x_js_framework_file"},{"key":"x_js_framework_file_fallback","value":"\/\/cdnjs.cloudflare.com\/ajax\/libs\/jquery\/1.10.1\/jquery.min.js","xtype":"textfield","namespace":"xsystem","area":"Site Scripting","editedon":"Jun 23, 2013 10:18 AM","name_trans":"JS Framework File Fallback","description_trans":"Fallback JS framework file","area_text":"Site Scripting","description":"Fallback JS framework file","name":"JS Framework File Fallback","oldkey":"x_js_framework_file_fallback"},{"key":"x_js_modernizr_file","value":"\/\/cdnjs.cloudflare.com\/ajax\/libs\/modernizr\/2.6.2\/modernizr.min.js","xtype":"textfield","namespace":"xsystem","area":"Site Scripting","editedon":"Jun 23, 2013 10:24 AM","name_trans":"JS Modernizr File","description_trans":"Path to modernizr.js file","area_text":"Site Scripting","description":"Path to modernizr.js file","name":"JS Modernizr File","oldkey":"x_js_modernizr_file"},{"key":"x_css_dev_enabled","value":"0","xtype":"combo-boolean","namespace":"xsystem","area":"Site Style","editedon":"Mar 27, 2013 08:40 AM","name_trans":"CSS Development Mode","description_trans":"Enable to show development CSS to admin users.","area_text":"Site Style","description":"Enable to show development CSS to admin users.","name":"CSS Development Mode","oldkey":"x_css_dev_enabled"},{"key":"x_css_file_dev","value":"\/assets\/css\/style-dev.css","xtype":"textfield","namespace":"xsystem","area":"Site Style","editedon":"Mar 27, 2013 08:01 AM","name_trans":"CSS Development File","description_trans":"Location of CSS development file.","area_text":"Site Style","description":"Location of CSS development file.","name":"CSS Development File","oldkey":"x_css_file_dev"},{"key":"x_css_file_pub","value":"\/assets\/css\/style.css","xtype":"textfield","namespace":"xsystem","area":"Site Style","editedon":"Mar 27, 2013 08:01 AM","name_trans":"CSS Public File","description_trans":"Location of public CSS file.","area_text":"Site Style","description":"Location of public CSS file.","name":"CSS Public File","oldkey":"x_css_file_pub"},{"key":"x_facebook_page","value":"https:\/\/www.facebook.com\/","xtype":"textfield","namespace":"xsystem","area":"Social Media","editedon":"Jun 23, 2013 10:23 AM","name_trans":"Facebook Page","description_trans":"Facebook page URL","area_text":"Social Media","description":"Facebook page URL","name":"Facebook Page","oldkey":"x_facebook_page"},{"key":"x_google_plus_page","value":"https:\/\/plus.google.com\/\u200e","xtype":"textfield","namespace":"xsystem","area":"Social Media","editedon":"Jun 23, 2013 10:23 AM","name_trans":"Google+ Page","description_trans":"Google+ page address","area_text":"Social Media","description":"Google+ page address","name":"Google+ Page","oldkey":"x_google_plus_page"}]}

If anything this is giving me an interesting view in how MODX/Ext works!

24,544 Posts

BobRay Reply #8, 13 years, 3 months ago

I think what that's telling us is that the getList processor (or the code it callls) is confused. It's sending a "total" of 28 and an array of only 25 items.

Both start and limit are correct in the header.

It's difficult to tell for sure, but I think the relevant code is in the getData() method in

core\model\modx\processors\system\settings\getlist.class.php

or (more likely) the listSettings() method in

core\model\modx\mysql\modsystemsetting.class.php

I notice that in the second one, $count is set before the limit is set, which might cause a mismatch between the count and the total retrieved, so if the limit is wrong at that point, you could get less than the total number.

Unfortunately, I don't see how limit could be changed unless maybe there's some plugin firing that uses $limit in its $scriptProperties array.

If you feel like digging deeper, you could put this code in the second file above line 34 (just above the $c->prepare(); line):

$xpdo->log(xPDO::LOG_LEVEL_ERROR, 'LIMIT: ' . $limit;

That should write the limit at that point to the MODX error log. It should be whatever your per_page value is set to.

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting

45 Posts

ram Reply #9, 13 years, 3 months ago

Hmm. I tried it, it just outputs LIMIT: 30.

24,544 Posts

BobRay Reply #10, 13 years, 3 months ago

Interesting. Assuming that I'm right about which file is executing, that would mean that getCount() is returning a bigger number of items that getCollection() called with the same criteria.

That would definitely be a bug, but I don't see anything obvious that would cause it.

Could you report what you're seeing as a bug at

http://bugs.modx.com

?

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting