[Module/Snippet] Content Getter : opinion and test

Hi,

I created a module / snippet for my personal needs : I wanted to supervise certain Web pages to know the updates of software like Firefox, Thunderbird, Grisbi, etc… without being obliged to go on each website.
By extending the module a little, there was possibility of recovering any part of the page. That can be very practical for collaboration, an advertising exchange of stringcourse or any type of information.
But, it is possible to recover contents without indicating the source, which is not very sympathetic for the author of the website.

It is for this reason that I post my work here and to know what you think about it.

All comment and point of vue are welcome.

Thanks

Guillaume

PS. If you think that I can propose it to the comunity, I really appreciate feedbacks (tests, translation, second reading...).

Best regards.


Changelog
Module
Version 0.1.1 (2006-06-23)
* URL is shown only if page title did not grabbed. If it is shown, it is truncated with a tooltip to see it completely. Else, the page title is shown and a tooltip permit to show the url.
* add column id on the main page to know ids for the snippet
* add format for the delay between 2 grabs (refreshing)
* update domTT to version 0.7.3

Snippet
Version 0.1.1 (2006-06-23)
* possibility to display custom list by specification of id
* possibility to force the visualization of the grabs
* now &onegrab accepts only id as parameter
* fix: chunk by default for list and onegrab
* add placeholder [+refreshing+]


2006-06-26 : I updated the module and the snippet to v 0.1.1. To update, you need to unzip and copy / paste folder in the .zip. After, go to the manager, run the module and, on the setting tab of it, add new setting (name: format_refreshing ; value : %H:%M:%S).

Thanks for your participation

2006-06-28: I renamed the title of the snippet / module to have an ethical name for an ethical use of it. I will rename the name of it after more feedbacks.

There are no feedbacks :’(
I think this means that I can make available this plugin. Without return from here to this weekend, I places it at the disposal.

Thanks

I tried it and it looked great, nice work! I didn’t understand though that for what is the "List" option used for? It said in the documentation that "List: an image (checkbox) indicates if the grab can be list (front end)", what does it mean?

Here’s a little suggestion if you have time: Put the ID’s of the "grab items" also visible (id_cg) and have a snippet to go with this, where user could pull the data from item/items by defining the id/id’s of the item/items in the snippet call. Possible to use chunk as a template for the output with placeholders for the fields (title, URL, last grab time, delay between grabs, old and new data)..

For the module GUI I’d suggest to remove the URL column and use the title for the link and show the link URL in a tooltip when user hovers the title text. Or if you want to keep the url visible in its own column, atleast truncate the visible url text to certain characters. If the url is very long, it screwes up the layout.

It would be good if someone who speaks english fluently would go over the language file and documentation. They need some work.

All in all, looks very usefull module and you have done clean work with the user interface, props for the "tooltips" popup!

I should be able to give this a blast tomorrow night if all goes well - I’m having one of those ’I can’t wait until it’s over’ weeks at work so just haven’t had chance to look at it yet.

I’ll post back with feedback - also I don’t mind checking over the language files and documentation as well

Quote from: doze at Jun 08, 2006, 01:46 PM

I tried it and it looked great, nice work!

Thanks
Quote from: doze at Jun 08, 2006, 01:46 PM

I didn’t understand though that for what is the "List" option used for? It said in the documentation that "List: an image (checkbox) indicates if the grab can be list (front end)", what does it mean?

With the snippet (and the right parameter) we can obtain a list of all the grabs wich have this option checked.
At the begining, the purpose of this module/snippet was to display only a list of content grabs from different websites to compare old version of the web pages with the current page. Thus, I can verify if new versions of many softwares are published without visit each page "manually".
Quote from: doze at Jun 08, 2006, 01:46 PM

Here’s a little suggestion if you have time: Put the ID’s of the "grab items" also visible (id_cg) and have a snippet to go with this, where user could pull the data from item/items by defining the id/id’s of the item/items in the snippet call. Possible to use chunk as a template for the output with placeholders for the fields (title, URL, last grab time, delay between grabs, old and new data)..

I hesitated for the id and I chose not to use it. Considering your request, I will add this possibility as soon as possible.
Currently, you can call a grab by its title (not page title !). This feature run only for one grab, not to create a list.

Quote from: doze at Jun 08, 2006, 01:46 PM

For the module GUI I’d suggest to remove the URL column and use the title for the link and show the link URL in a tooltip when user hovers the title text. Or if you want to keep the url visible in its own column, atleast truncate the visible url text to certain characters. If the url is very long, it screwes up the layout.

It is ok personally I did not have this problem. This is why I asked for testers
I will combined the 2 solutions. The page title is grab from the webpage when the first grab occured : I can’t display the page title if it is non-existent. If I have the page title, I display it with the full url in a tooltip, else I display the truncated url with the full url in a tooltip.

Quote from: doze at Jun 08, 2006, 01:46 PM

It would be good if someone who speaks english fluently would go over the language file and documentation. They need some work.

Why !! My english is so bad !!
It is the 2nd reason of my post here I think I can translate it in french
I know my english is not brilliant As I had much text to write (and that it is difficult enough for me), I used Google (without reading again afterwards) . I will be really grateful if somebody could look to these writings.

If someone would to translate in other language, I will be charmed to be able to integrate these files but I think it would be better that I code the features above.

Quote from: doze at Jun 08, 2006, 01:46 PM

All in all, looks very usefull module and you have done clean work with the user interface, props for the "tooltips" popup!

For tooltips I used domTT
For the top menu (with buttons "Cancel", "Documentation", etc...), I wrote a php class to create module easly. This class can also format error string, manage the language files (for the module) and more I coded a class to manage language files wich it is used by the library.

Quote from: garryn at Jun 08, 2006, 01:52 PM

I should be able to give this a blast tomorrow night if all goes well - I’m having one of those ’I can’t wait until it’s over’ weeks at work so just haven’t had chance to look at it yet.
I’ll post back with feedback- also I don’t mind checking over the language files and documentation as well

Thanks Garry

I published a new version of ContentGrabber. If you can test it I will be very happy

All feedbacks and participations are welcomed. Thanks for your help

You asked for feedback on the ethical issues of grabbing content without permission.

If you do not wish people to consider that the point of your module, perhaps you could consider retitling the module - ContentLoader, WebpageImporter, WebpageChecker, LoadExternalPage or something.

To "grab" does infer taking something without permission or consideration.

Personally, I’d quite like to import comments from a syndicated version of my site’s RSS feed, and I will be looking at your module to see if that is any help.

Quote from: PaulGregory at Jun 27, 2006, 12:00 PM

If you do not wish people to consider that the point of your module, perhaps you could consider retitling the module - ContentLoader, WebpageImporter, WebpageChecker, LoadExternalPage or something.

To "grab" does infer taking something without permission or consideration.

Ok. I did not know that this had this precise meaning. I thought that was simply to take contents elsewhere. Thank you very much for this information
Currently, I don’t know how rename it but I will do it when I will have found a name. It is difficult for me to found a good name because I’m french and my english is very bad (everybody can confirm it !). Moreover, I will like to find a name which makes pass the goal of this combo: to get contents (partial or not) of another website but without concept of rob (?) (as you did noticed it to me). I do not know enough the nuances of each word.
Quote from: PaulGregory at Jun 27, 2006, 12:00 PM

Personally, I’d quite like to import comments from a syndicated version of my site’s RSS feed, and I will be looking at your module to see if that is any help.

I’m sorry but it is not possible (or I don’t understand what you want ). This resource is to "import" content from a website using regular expression.
This resource open a connection on the URL of the website and get the (x)html code. After, this code is parsed with regular expression to get only that we want and return the result.
The original is truncated but not modified, excepted (only if you want) the img tag to permit to display the image. It can be useful if you get a link wich contains an image. I found this problem when I tried to get the version and the link to Mozilla Thunderbird on the official page.


PS: contentGetter it is a good name ?

To be fair, "grab" might not have the same meanings in all communities, and the primary meaning is still to quickly get something. The "before someone notices" bit is not a precise interpretation.

Anyway, ContentGetter is an excellent name, especially as it reminds one of a HTTP Get Request.

Thanks for you answer
I will wait of another feedback, then I will modify the name.

I rename the title of the topic.