Problem with static elements in my S3 Bucket in AWS

Hello!

I’m trying to make static elements in my S3 Bucket in AWS.

I followed all the documentation: I put my Keys, changed my bucket URL as pointed in other forum threads (https://forums.modx.com/thread/88840/connect-to-amazon-s3-media-source#dis-post-488624) and now I can see my Files in the bucket. I can upload new files, download, delete and edit perfectly.

But still I can’t make static Elements (Templates, chunks, snippets..) in the bucket.

I reinstalled a clean version of modx to be sure that is not a plugin that messes with the Media Sources.

When I change the Static File route to the AWS file and try to save, it shows an error:

“An error occurred while saving the snippet.”

In the System > Error Log shows:
[2014-06-06 09:17:49] (ERROR @ /index.php) Could not retrieve data to cache for resource 1
[2014-06-06 09:17:54] (ERROR @ /connectors/element/snippet.php) An error occurred while saving the snippet.Array

(resource 1 is the snipet called test in my clean installation of modx)

Also doesn’t load the content when I try to access the Element pointed to the AWS bucket. (via static file in Elements, remember that if I access via File, then I can edit, remove, upload..)

One of my other tests was to upload my application to my AWS account and try to access the bucket from there (to discard possibly firewall or connection issues), but I got the same result. I can list the tree, but I can’t make static elements.

Do you have any tips or suggestions? Maybe there is something that I miss!

Thank you,
Jordi

PS: I have attached 3 snapshots with the correct File tree listing (from the bucket), the Media Source config. and the error on save the static snippet Element.

I'm not so sure that static elements can work from remote URLs like that; I think they may require a file path, and thus can only work on the server filesystem. I may be wrong about that, though...

See the getSourceFile() function on line 448 in core/model/modx/modelement.class.php

You might want to test out the S3 media source using images and a TV. If that gives you trouble, then you know it's something else.

Thank you for your time sottwell!

It makes sense. I didn't think about that. I suppose you can't store code in a separate environment than your application. It helped a lot!

I'll use the bucked for media (img, vids..) and docs (pdf, doc..)

Again, thank you.
Jordi

Well, like I said, I'm not real sure about that, so someone else may be able to shed more light on the subject. I suppose I can go and run some tests on my known working S3 media source. I'll get back to you on what I find.

Yeah, I get the same error when trying to use a sample .html file as a static chunk source.

I see a function isStaticSourceMutable() that uses if (file_exists($sourceFile)), so it can only work on the local filesystem.

A little research turns up some interesting ways to possibly take care of that. Instead of file_exists, it could use cURL, or it could use get_headers($filename) to see if it returns anything other than a found code.

Thanks for reviewing that sottwell!

And thanks to find a possibly way to solve that.

I think I had a bad approach with my design of the application.

It can be done, but it's probably better to make the snippets, chunks and TV static in the file system. So we can put them in the version control system.

Maybe in the next Modx Manager update you won't be able to choose the AWS Media source for this kind of Elements.

Again, Thank you!
Jordi

Well, this touches rather closely on a project, or rather a set of projects, I'm working on. First, creating a SFTP driver for media sources (media source type), then creating a custom modSourceStaticResource class that will allow the static resource to use a media source, local or not. Then we can have .pdf or other documents on an external server, accessed with SFTP by a static resource. And finally, a script (snippet or CMP) that will take a given location (local or remote directory) and generate static resources out of the files from that location.

It's been a rough road so far, but I am learning a lot about the inner workings of the core, and how static stuff and how media sources work.

I would love to try out that new sftp media source
I'm really amazed at how flexible the development environment can get with dynamic media sources and external ones.
Already using a dynamic media source for creating multiple thumbs on the go for MIGx custom gallery (custom object folder and multiple thumbs generated on the go when creating a new custom object)

External and dynamic media sources can improve and even lower maintenance costs con projects with heavy media usage.
Separating media files from the running environment has a lot of benefits.