Slow page loads in Revo 2.2

Hello All,

I've got a strange problem with a site I've been building in Modx Revolution. I have two identical (as far as I know ) servers, both running:

Ubuntu 11.10 (GNU/Linux 3.0.0-12-generic x86_64)
MySQL 5.1.63-0ubuntu0.11.10.1 (Ubuntu)
PHP Version 5.3.6-13ubuntu3.6
Modx Revolution 2.2.1-pl

I took our developement version and moved it from the dev server to the prod server, and everything runs, but page loads for the application are incredibly slow, as in over 1 minute.

Load times for everything besides the application are equivalent -- the modx manager runs the same on both servers. Loading a test page that calls phpinfo is equivalent. Database response times are equivalent.

I'm at a loss here, any ideas? I surfed these forums looking for "slow resposne" yesterday, and I found many hits, but they don't apply to my situation. BTW, I migrated the site following the steps in the modx documentation:

http://rtfm.modx.com/display/revolution20/Moving+Your+Site+to+a+New+Server

Finally, I should point out that I did not do the initial configuration of the production server. But I spent a lot of time checking the Apache, PHP, and MySQL servers and they seem fine.

Help! Any ideas, thoughts or clues deeply appreciated.

-- Mark

Hello Again Everyone,

I spent some more time looking at the problem, and I think its permissions!

Here are my results:

First, I wiped and rebuilt the server. I made careful notes, and made sure I had the LAMP stack performing properly.

Then, I migrated the site again. After restoring the databases and unpacking the modx directory, modx ran very fast! But the snippets wouldn't run. Pages and chunks were displayed using the correct template, and FURLs were working. AJAX calls to my REST API *would* run php, so I could actually log into the site.

However, none of my logging was running, due to permissions problems. So then I started playing with the permissions and ownership of the files...

By changing the owner of all the files to www-data the site will run, but again very slowly. By changing the permissions on the modx/core, modx/connectors, modx/php (with my custom PHP objects) and modx/logs the site would run, but again very slowly.

I am at the point where I keep replacing the modx directory with my original tar source, trying things, and then removing the modx directory and copying it over again. I'm reviewing the points about permissions.

Also, I know I've not done things quite the "modx" way. I made the mistake of putting my assets into folders directly underneath modx, instead of putting them in modx/assets. Also, note that I am not using modx's connectors, I have a stand-along REST API built.

But I am close, and I will post again when I get it working. As always, any ideas/comments/observations are welcome, and I appreciate the 33 views I've had so far, even if no one has posted a response.

-- Mark

Hello,

Could you tell us more about the speed issue, is it page or site specific? Does it affect the manager? How do you have apache configured to interact with php (ex: suphp, fast-cgi, dso, etc)?

Okay,

Again, the speed issue only affects the main website. In particular:

-- The manager runs fine. Page load times are pretty much the same between the dev and prod servers. The manager runs immediately after I copy in the modx directory.
-- PHP pages by themselves run fine. Load times for a page that just calls phpinfo() are the same
-- HTML page run fine. Load times for a page that displays an image and simple html are the same

The webserver is Apache 2 running under Ubuntu linux. PHP is run via the php5 module in the mods-available directory underneath /etc/apache2. All of the software was installed using the Ubuntu Package Manager and apt-get.

There is more I should post, but unfortunately I'm out of time right now. I'll post again tomorrow with the steps I take after copying in the modx directory from my tar archive.

Thanks for the reply!

-- Mark

Do you have complex getResources calls on the pages? That's the most common cause of slow page loads.


------------------------------------------------------------------------------------------
PLEASE, PLEASE specify the version of MODX you are using.
MODX info for everyone: http://bobsguides.com/modx.html

The last time slow page loads plagued a site that was fine on localhost, moving the javascript to load below the footer area sped it up considerably.

Hi Guys,

Its not permissions. I finally got some more time to look at this problem again, and I have some results to post. As well as some replies:

@BobRay -- I don't think so. I don't even know what getResources is, but I think its a method on the modx object?

@Frogabog -- I use jQuery's document.ready function. I do have inline javascript code on a few pages, and it would be hard to move it after the </body> tag, because its dynamically generated by snippets. Also, all my configuration uses 192.168.x.x values, dev is 192.168.1.30 and prod is 192.168.1.10. Config.inc.php uses 192.168.x.x as well.

Today I migrated the site again, and I'm still gettting very slow load times on prod. Here's what I've done and what I've found:

I deleted the modx directory at /var/www. I made a new tar of the /var/www/modx directory on my dev server, and new dumps of the modx and client databases.

Previoulsy, I had unpacked the tar in my home directory, then copied it over to /var/www using sudo. This is why the permissions and ownership on the files was wrong. This time I copied the tar and unpacked in /var/www using root. This restored the permissions properly, including ownership of files and directories.

I logged into mysql as root, and used the source command to reload both the modx and the client database. I edited core/config/config.inc.php, changing 192.168.1.30 to 192.168.1.10 on the line for $http_host.

Then I tested the site again. My snippets ran, but it still takes 30 seconds for my main login page to load on prod, vs. 0.25 seconds on dev. I logged in on both sites and compared the results, and found that the REST API login routine on prod ran 4 times, taking 10 seconds where as it ran only once on dev taking less than a second. Strangely enough, everything works on prod, I can login, use the site, etc., but very slowly.

I have a REST API, which (as I mentioned in previous posts) runs fine on both machines. I use jQuery to make an AJAX call to a stand-alone PHP page in the main modx directory, which returns results as JSON strings. I've written a pure HTML/Javascript testing page that makes the same calls, and it runs quickly on both dev and prod. The REST API login routine runs just once from this page.

So I'm still left with a conundrum. Why do I see four calls to my login routine on prod, where I only see one on dev? I'm going to have to turn on more of the debugging to see exactly why this is happening.

I have to say this must be my code. I've only made one other site using Etomite, and the publishing process was basically the same. (BTW, that site was a success, I'm sold on modx.) I know that Revo 2.2 has its own system to handle AJAX calls, but I haven't had the time to study it, and I went with what I knew. Still, I can't see any reasons why this shouldn't just run "out of the box" after migrating it.

I'll post again when I get more results, or fix this. And I will fix this. Again, any replies, questions or suggestions are appreciated...

Signed,
Slow Loading Mrex

Don't know if this pertains to you, but:

I have run into issues with load times when migrating data. Even if everything was the same, something about the production database environment must have been different from the dev environment. I found that I had to make a Snippet to re-save() all of my Resources, Templates, Chunks and Snippets (with all of their current values) after migration. After running it and refreshing the cache, all of my load issues were solved.

This eventually led to me getting rid of the dev environment completely because I couldn't completely replicate the production environment. The production database was housed on a remote server by my provider, so I also did not have control of it. I no longer have this problem, but I believe the issue lied with some difference between the environments.

Aha!

I got some time today to continue investigating and I have more results. First, database calls to the client database are taking 5 seconds on prod, less than a second on dev. Second, my loginUser command gets sent twice to the REST API on prod. I turned on tracing in all snippets, and then logged into both dev and prod. I then analyzed the logs side by side using Beyond Compare (windiff could also have been used).

Obviously my next step is to figure out what's causing database calls to take 5 seconds on prod. But I also want to find out why my loginUser command gets called twice.

@fuzzicallogic, thanks for the reply. Can you give me more information on how you created the database when first migrating the site? Did you just use CREATE DATABASE from a mysql command line? Also, did the manager run slowly before you re-saved all your modx elements?

BTW, I cannot work without a separate dev and prod environment, I can't develop while users are using the site. But I should also tell you that when I did that etomite site, publishing was fast and easy. Literally, I dumped the modx and client databases, tared the site, copied the files, untared the site, and then use SOURCE to restore the databases, as the modx migration page says.

One clue might be that when I dump and the restore the client db, the stored procedures (routines) never get saved. I then use MySQL Workbench to synchronize the prod client db's stored procedures.

I will now carefully examine the differences between the client db on dev and prod. Hmm....

Enough for Now,
-- Mark

P.S. I've attached the logs if anyone is interested.

@mrex2mark:

Did you just use CREATE DATABASE ... ?

Unfortunately, no. My host requires that databases be created via their control panel. I have shared hosting via 1&1. In other words, I don't have any direct command line interface. I don't know how they did it. I exported the data & structure via command line and then had to import them via myPhpAdmin.

Also, did the manager run slowly before you re-saved all your modx elements?

Everything ran slowly. At one point, I had to wait 5 minutes per request. Reloading the contexts did not help for me. I literally had to make a script to go through every Resource and Element and save() them individually. In order to keep the data the same, I used xPDO grabbed the item, copied the variable, overwrote the placeholder with the copy, and then save() it. It fixed all of my Resources and Elements.

The issue seemed to be with the way the data was exported/imported. I'm guessing it was a slight encoding difference (probably cr/lf, honestly). My dev environment was Windows and the prod is a linux server.

I cannot work without a separate dev and prod environment.

I don't recommend it at all. I did so because my development methodology allows me to work while users are using. It was extremely tough to adjust to and I don't recommend it to anyone. I just got tired of the issue with migration. If I had more control over my prod environment, I would not have had the issue.

One clue might be that when I dump and the restore the client db, the stored procedures (routines) never get saved.

That is an important clue. It should be saving your stored procedures completely. If those aren't being saved, what else isn't being saved?