2.7 and php-7.3 performance issue with finfo

2 Posts

jscaltreto Reply #1, 5 years, 2 months ago

Since 2.7, modFileHandler->isBinary() uses finfo to check the mime type of files (https://github.com/modxcms/revolution/commit/938add62da91335a5c4ebb8412fbf239fa8efcd7). This happens when listing files from a media source (such as in the files tree or browser).

After upgrading to 2.7 on php 7.3.1 I noticed it taking several seconds to populate file lists when before it was instantaneous. The slowlog pointed me to the call to finfo->file() in modFileHandler->isBinary(). When I commented out the finfo stuff the file lists populated instantly again. But, here's the kicker, I downgraded to php 7.2 and the problem went away. It seems php-7.3+ has an issue where finfo->file() is taking 30%+ longer than in 7.2, though I've observed much longer delays on an even moderately loaded system. The script in this gist (docker required) illustrates the issue: https://gist.github.com/jscaltreto/8f6adf38e7abb08a831ef37b84c14ace

On a prestine DigitalOcean droplet with nothing else running:

./test.sh 1000
Testing finfo over 1000 iterations
---
Testing PHP 7.2
Took: 1.2104759216309
---
Testing PHP 7.3
Took: 2.4543979167938

The test took more than twice as long with 7.3. On my machine which considerably more load, you can start to see why this could be a serious issue (note this is only 100 iterations, equivilant to a directory listing with 100 files):

./test.sh 100
Testing finfo over 100 iterations
---
Testing PHP 7.2
Took: 0.10123181343079
---
Testing PHP 7.3
Took: 34.721122980118

So, if you're seeing file lists take a long time to load and happen to be running revo 2.7 on php >= 7.3, be advised. If others care to verify my result or have other thoughts on what could be going on, I'd be interested is hearing it. I filed a bug for php (https://bugs.php.net/bug.php?id=77536), but unclear when/if anything may be done.

If this is a verified issue, I wonder if modx devs would consider adding a php version check to bypass the finfo call in php 7.3.

This question has been answered by jscaltreto. See the first response.

24,544 Posts

BobRay Reply #2, 5 years, 2 months ago

Thanks for running that down and filing the bug report.

Did I help you? Buy me a beer
Get my Book: MODX:The Official Guide
MODX info for everyone: http://bobsguides.com/modx.html
My MODX Extras
Bob's Guides is now hosted at A2 MODX Hosting

discuss.answer

2 Posts

jscaltreto Reply #3, 5 years, 2 months ago

It seems as if the issue is related to Transparent Huge Pages in linux; specifically when the kernel needs to defrag memory to allocate. Changes in php 7.3 seem to make use of huge pages during finfo calls, and in some cases this can lead to the kernel initiating a memory defrag on every call to finfo->file(). This can add a ton of latency to modx file listings on systems that have heavy memory fragmentation.

There is a workaround if users have root access to the host on which modx is running, which involves disabling defrag on every madvise(). If you're seeing the same issues I was encountering on php 7.3, first check if THP is enabled and set to defreag on madvise():

jake@new-vps ~/phptest % cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
jake@new-vps ~/phptest % cat /sys/kernel/mm/transparent_hugepage/defrag 
always defer defer+madvise [madvise] never

You can change the defrag option to "defer""

echo defer | sudo tee  /sys/kernel/mm/transparent_hugepage/defrag

On systems with heavy memory fragmentation this seems to eliminate the latency in these calls. In my case, the effects of this tweak were considerable:

jake@new-vps ~/phptest % cat /sys/kernel/mm/transparent_hugepage/enabled
always [madvise] never
jake@new-vps ~/phptest % cat /sys/kernel/mm/transparent_hugepage/defrag 
always defer defer+madvise [madvise] never
jake@new-vps ~/phptest % ./test2.sh 100                                 
Testing finfo over 100 iterations
---
Testing PHP 7.2
Took: 0.11759901046753
---
Testing PHP 7.3
Took: 30.368657827377
jake@new-vps ~/phptest % echo defer | sudo tee /sys/kernel/mm/transparent_hugepage/defrag   
defer
jake@new-vps ~/phptest % ./test2.sh 100                                                  
Testing finfo over 100 iterations
---
Testing PHP 7.2
Took: 0.096560001373291
---
Testing PHP 7.3
Took: 0.44092488288879

This won't survive a reboot, but some sort of init script should be doable. Users who don't have root access may be out of luck other than to ask their hosting provider.

Answered 2.7 and php-7.3 performance issue with finfo