Upgrading GetFeed ?

Hi,
I am using the add-on GetFeed, a simple snippet (less than 50 code lines) to parse the feeds from 5 different blogs on one page (Modx resource).
Works very well except when one of the feeds is out (blog down for maintenance(503), error 500, time out, etc)
The drawback with GetFeed as currently written is that you get a blank page for the whole resource i.e. cannot see the other parsed feeds which are OK. Instead we should get blank lines only for the url/feed not working.

When looking at the code (lines 23-24 of the snippet) :


What could be added to test the $url first, and if not working, assign $url=''; (empty) ?

Any idea ?

Thanks !

PS : I have found an old script i had done some 10 years ago :

with this it would be easy to check the feed :

I need to understand (i have lost touch !) what is behind i;e. function ? class ?? where ? [ed. note: justinet last edited this post 8 years ago.]

Are you calling getFeed uncached five times?



Looking at the code, it looks like that would do what you want.

The parse method is in core/model/modx/xmlrss/modrssparser.class.php

The function it calls is in core/model/modx/xmlrss/rssfetch.class.php

Quote from: BobRay at Sep 22, 2018, 12:59 PM

Are you calling getFeed uncached five times?

[[!getFeed ...]]

Yes, believe it's necessary since the feeds can change when new posts are added ?

Thanks a lot Bob for advising the locations of the method/function, will save me a fair amount of time !

Since they are all independent, uncached tags, I don't see how one empty one would affect the others unless it's somehow corrupting the Magpie feed parser itself. Even then, it should only give you a blank page if the empty feed is the first one.

Do you see anything in the MODX error log?

Have you watched the Network tab in Chrome Dev. tools (Ctrl-shift-i)?

Hi Bob,
When a feed is not formed correctly i get this error message : " core/cache/includes/elements/modsnippet/4.include.cache.php : 46) Error parsing RSS feed at https://www..." and only blank lines for this specific feed, so usually working at it should !

However in one case when i got the blank pages (had all error messages turned off since moving the site to production), i removed all the feed urls one by one to spot the culprit and found one problematic feed. The page rendered OK when i just removed its url. When checking with the W3C feed tool : i got a "time out" message. Not only the feed but also the whole blog wasn't available (error 500 as far as i remember). Now this blog is on again with its feed working so i cannot replecate the corresponding error.
Nothing related found in the site server error log. In Modx core cache error logs i get : : working at it should when a feed is not good, but it's for other feeds, not that one !

I had errors
due to the use of some old code in Breadcrumb, but that's not related and now fixed.

I also got this error (which in all honesty doesnt tell me much, don't know how this xpdo class works) :

that last one though could be related with this blank page event. (timewise match). My site is all UTF-8.


My concern being that if one of the feeds is out again the same way, i get these blank pages instead of blank lines for the related feed.

Many thanks again for your well appreciated feedbacks !



PS: Additional info which could confirm this last error message is related :

When looking at the site triggering the feed issue, i can see this message with the chrome tool :
A parser-blocking, cross site (i.e. different eTLD+1) script, <URL>, is invoked via document.write. The network request for this script MAY be blocked by the browser in this or a future page load due to poor network connectivity. If blocked in this page load, it will be confirmed in a subsequent console message. See <URL> for more details.
scriptaculous.js?ver=1.9.0:31 A parser-blocking, cross site (i.e. different eTLD+1) script, https://ajax.googleapis.com/ajax/libs/scriptaculous/1.9.0/builder.js, is invoked via document.write. The network request for this script MAY be blocked by the browser in this or a future page load due to poor network connectivity. If blocked in this page load, it will be confirmed in a subsequent console message. See https://www.chromestatus.com/feature/5718547946799104 for more details. [ed. note: justinet last edited this post 8 years ago.]

Did you install your MODX site with Scriptaculous? I've seen quite a few reports of trouble with Scriptaculous installs over the years.

I don't know why it would be involved in a retrieving or parsing an RSS feed. You might look at your index.php and root .htaccess to see if Scriptaculous has injected an update check somewhere.

I can see why you may be getting the blank page. If the feed doesn't respond, you'll pass the PHP default 30-second timeout, which will throw a fatal error. You might see if getFeed lets you set your own, shorter timeout, which might abort that call more gracefully. If there's no timeout property, getFeed probably uses cURL, so you could probably change its code to set a timeout at say 10 seconds.

I'd also take a look at your DB in PhpMyAdmin (or let SiteCheck do it for you). You need to look at the structure tab to check each table and each text field within each table. I'm betting you have a latin1 collation somewhere. If so, see this: https://bobsguides.com/convert-db-utf8.html.

Hi Bob,
Sorry for my misleading post above. Scriptaculous and the latin1 collation are on the blog with the feed causing trouble, not on my site.
The chrome message i posted is when visiting this blog site, not my MODX site.

Changing the time-out for GetFeed sounds good though ! As long as this is below this 30 seconds threshold this should be OK, right ?
(doesn't seem wise to increase this 30s in the server php.ini)


Dont really know how to pass an optional time-out parameter to GetFeed ?
Like this : ?
[[!GetFeed &url=`feed_url`&offset=`0`&limit=`10`&tpl=`tpl.GetFeedInt`&magpie_fetch_time_out=`20`]] ?? since the snippet line 23
is calling modRSSParser class using RSSfetch classes

Apparently it's using a php class called Snoopy, somewhat similar (i believe) to Curl to retrieve distant info.

Anyhow (line 390 of rssfetch.class.php file)
there is a "MAGPIE_FETCH_TIME_OUT" already set up to 5 seconds,

so if i increase this to say 20 seconds this can only make this issue worse ??

Thanks [ed. note: justinet last edited this post 8 years ago.]

There are two possible timeout settings. One sets the timeout for the initial connection. The other sets the timeout for the whole cURL operation. It's hard to know which one the MAGPIE_FETCH_TIME_OUT is without looking at how it's used, though setting it longer can't hurt anything.

FWIW, I like to have my PHP timeout set to 120 seconds for longer-running extras like SiteCheck and Notify. That would be the first thing I'd try.