Internal Server Error on Manager Panel

Hmm. Is it MODx spiking memory use, or is the host just severely overloaded, or has the hosting company configured their servers to limit memory use for any one user? That would certainly be a good way to increase sales of VPS and other upgrades.

I’m wondering that myself. Because upgrading to VPS was basically like their second recommendation other then optimizing the site (which isn’t that big to begin with nor is it all that advanced or traffic heavy).

Whatever is going on happened recently, because I was working on things last month with no problems whatsoever.

Strange little development. I was taking a look at the memory usage (shows things like memory, CPU usage, etc) for our account and there was a spike in memory that hit the limit around 8:00pm and I haven’t touched or logged into the manager for Modx since like 6.

Now you’re getting somewhere. Check the server logs for that same time and see what kind of access was reported. You may be getting hammered by some kind of hacking or DDOS attempts.

And we probably shouldn’t be so quick to think cynically about things; if you were using something like a chat or forum application that was getting busy, then a VPS is the solution.

You’re right, I’d like to give our host the benefit of the doubt that it’s not their doing.

The site doesn’t have chat functions or forums. Even our blog doesn’t see much action other then spam comments. So we can rule that out.

Oh boy I took a look at the logs and there’s some serious things going on here. This is what I found:

From 6:27 to 11:21 the site was accessed 42 times from a IP in Chicago the organization was listed as SingleHop.

From 8:22 - 9:38 the site was accessed more then 100 times (I lost count over the pages) from a RedStation Limited which is located in the UK. (I’ve seen RedStation related IPs come up multiple times this week)

I’ve also seen several a few days ago from a Russian located IP.

There was a few other multiple IP’s accessing the site throughout the evening and on the resources log there was another spike that hit the limit at around 9pm. So that’s two spikes in two hours not related to me doing anything with Modx’s manager.

I’m wondering if they’re search engine crawler bots, like for example with say Google just scanning through the site.
The thing is I do have a robots.txt set up and I think it’s excluding everything it should but now I’m wondering if I have the directories listed correctly in the file. I’m seeing now the url directories that these IPs are going through are my assets folder, my images folder, my css files, etc.

Just in case here’s what my robots.txt looks like:

If someone is running a bot designed to look for known vulnerabilities in common applications (such as previous version of MODx with known vulnerabilities in some of the add-ons) it will ignore your robots.txt file.

What were these agents requesting? I’ve been hammered by user agents requesting a specific file in dozens of locations that made it clear it was searching for a vulnerability in an application I wasn’t even using (WP, I think it turned out to be if I remember correctly; it was some years ago.)

Also, a Google search on the offending user agent may turn up some interesting information, if it is a bot of some kind.

RedStation appears to be a hosting company specializing in dedicated servers and colocation, so it’s probably not they themselves beating up on you, but one of their customers (or one of their servers has been taken over by a botnet). You might be able to get somewhere by contacting them. http://www.redstation.com/

SingleHop is another dedicated server/cloud server hosting company, so the same would apply to the hits from them. http://www.singlehop.com/

I tried to narrow down how much by who and what was going on yesterday and here’s what I found:


Brooklyn NY appeared 279 times, looked all over the site and is connected to http://www.networksolutions.com

Woodlands CA appeared 98 times, all over the site, connected to Net2EZ and Reachlocal.

UK appeared 219 times, all over the site, and this is by the Redstation IP.

Chicago IL appeared 84 times, looked at the two main rss feeds from the WP blog over and over again through out the night. and this is by the SingleHop IP.

This is all from about 6pm to about midnight or so yesterday.

Just recently (1:37 am) someone from PSINet (associated with Sitebuildit.com) tried to access an .htm that I know for a fact doesn’t exist.

I think I’ll try contacting some folks like you suggested if I can today and see if I can get anything.

Some of those are probably just badly configured clients, like the feed access. If I’m developing a feed display on a site, it might at first be configured to refresh at an odd interval, but as I finish up it will be set to a more sane interval.

It’s possible to deny web request access to most of your site using the .htaccess file, denying everything by default and then explicitly allowing access only to the index.php files and the images, js and css files (these are the only ones that need to be requested from the outside). This won’t stop the actual requests, but at least it will protect the files from access. Most of the files in MODx will return immediately if they haven’t been accessed properly (usually included by a calling script within MODx), but there’s no need to let anybody access them at all when the only use they have is to be included by other internal PHP scripts.

I wouldn’t mind doing that for a little extra protection. How would I go about doing that in the .htaccess file?

I have two questions real quick about the .htaccess file if you don’t mind. I have two of them, one in my main directory and when I look at it, it has several statements, some commented out. And then there’s one in my manager folder which has the few statements we talked about earlier in this post.

What’s the difference between those two?

Also they’re both named ht.access rather then .htaccess, is that an issue?

The files that work are named .htaccess. The ht.access files are just for examples. For the most part, they can simply be renamed to work, but not always. That’s why the default installation doesn’t name them .htaccess in the first place. For example, if a host supplies its own .htaccess file with required settings it would be best to simply copy the relevant rules from the MODx ht.access file into the hosting-supplied .htaccess file.

The web server reads these files as the first thing it does when a page is requested (in the case of MODx, it’s the index.php file). The instructions in the file act as run-time configuration, telling the web server how to behave for this particular page. These files are inherited, so any rules in the web root are also applied to any pages requested from inner folders. For example, the rules in the web root will apply to the manager/index.php file. That is why the one in the manager folder turns off the rewrite engine, as we don’t want its URLs rewritten. It may also have the rule to turn off (or modify) the security settings, as we need to be able to have the POST from our Manager editing forms processed without getting blocked, but this rule isn’t necessary in the front-end. The rules in the /manager/ folder’s .htaccess file will override any that were inherited from the .htaccess file in the web root folder, so turning off the rewrite engine stops it from checking to see if it needs to rewrite the Manager’s URLs.

The later version of MODx doesn’t need the one in the manager (except for turning off the security feature if needed), since one of the rules in the web root .htaccess file specifies that the rules are not to be applied to the /manager folder, or to images and a few other odds and ends that might be broken by applying the rewrite rules to them.

You may not be able to see the .htaccess files, as files that begin with a . are considered system files and are normally not displayed in a file listing. Your FTP client can be configured to show hidden files. All UNIX-based systems, such as Linux and OS X have a great many of these hidden files, and the Apache web servers follows the same convention of using . files as its hidden configuration files, such as .htaccess and .htpasswd (which contains login information for password-protected pages and sections of a website).

Here’s a good reference for a rule for protecting all .php files; you would need to make another one explicitly allowing access to index.php files for the site to work.

http://stackoverflow.com/questions/1340001/deny-direct-access-to-all-php-files-except-index-php