Discussion: Less restrictive form validation

Yesterday I had a interesting discussion with one of my customers. I was about freedom of choice and verification of forms...

The baseline is that sometimes input verification is a VERY BAD THING.

Example. I live in a city where groups of buildings are adressed "Q4,10" rather than "Q4 Street, Number 10" and one of my friends lives at a very remote place called "Einsiedlerhof". No street name, no house number.

I could post countless forms, where I get an error message time an time again.

"You have to fill in a correct street name (minimum 3 letters)" or "please fill in your house number". Screaming, crying and begging wont help, I can’t submit the form without tweaking my  address or adding some rubish to one of the required fields. So Q4 becomes "Quadrat Q4" and my friend has to fill in house number "1" where there is no number.

After some brainstorming we came up with the idea, that rather than denying a visitor to send a form, it would be much better to ask for some corrections and if he is sure, that everything is right, to let him continue.

After submitting a form with "invalid" content, it should be returned with a message telling the visitor about a possible conflict. Next to the submit button a check box could be added labelled "I know what I’m doinig, please let me continue...". Checking the box and hitting submit again, would finally submit the form, possible error non-withstanding.

This would be a much more user friendly scenario, wouldn’t it?

Regards,

pepebe

P.S. I just added a few lines to eform.inc.php to give valid form fields the class "valid". If you are interested to add this feature to eforms, you just have to add a single line of code to the file.

Instrucions: Backup your original file (better save than sorry). Then look for the lines of code below (somewhere around line 250) and add "if(!$rClass[$name]) $rClass[$name]=$validClass;" right above the "end required test" comment.

The result should look a bit like this:



Now open your eform snippet and add under "// eForm Params" the following line:



Save and test the whole thing. If you have a problem, restore your original eform.inc.php and send me a PM...

Have fun!

Surely the issue is whether you need to actually validate those fields and whether the validation is appropriate. Not that this isn’t a cool idea. It might work really well for things like email addresses where our validation sometimes falls down. But I do think most of the problems could be overcome by thinking properly about how and whether you want to validate a field in the first place.

I don’t agree.

I believe form validation should be something offered to enhance the user experience. Assisting, rather than restricting. Validation rules can prevent many simple error and are therefore generally helpful.

Forms are a crucial part of any kind of user interaction. I can’t remember how often I have turned away from a commercial site, because they used used some kind of downright stupid form validation that annoyed me with some generic rules that just didn’t fit with me being an individual.

The message I get from that kind of user interface is:

"We don’t need customers too stupid to fill in our clever forms".

I wager, that about 1.000.000 Euros get lost EVERY DAY because of simple things like that.

My opinion is: Most of the time, validation is a nice thing to enhance the user experience. There might also be situations where strict validation would be necessary. So I want to have the choice between a "soft" and a "hard" validation.

Yet, right now, eform validation is totalitarian. Its either follow the rules or suffer the consequences. If the validator finds an error, there is nothing you can do against it, but an unconditional surrender.

"Sir, grant freedom of thought!"

Regards,

pepebe

Yes, I agree it is a COMPLETE waste of time when a web form won’t handle legitimate data (I’ve wasted HOURS with AT&T trying to change my address to include a "1/2" at the end of my house number) but form validation is here to stay. The bottom line here is simply security, and MODx users should be very aware of how a form can be blasted apart. If you’ve ever lived through a SQL injection attack this lesson REALLY sticks with you. Nothing is more horrifying that looking through your database rows and seeing SQL statements where names and phone numbers are supposed to be.... and you wonder how badly your data has been compromised.

Fun little facts:

• MODx Evolution does not use mysqli; it does not use prepared statements.
• Many of the user-submitted Snippets parse together SQL queries using string concatenation.
• The standard MODx database allows characters where arguably there should only be integers (e.g. phone number)

All of this is to say that MODx has SQL injection vulnerabilities when it comes to forms. I love MODx, but you can never be too careful when you have a web form... it can be a vulnerable little portal into your valuable database, and you can and should carefully filter the data that’s entered on those forms.

I wrote an article about this a while ago outlining some good practices:
http://www.fireproofsocks.com/php/preparing-mysql-statements-in-php-5/

I believe form validation should be something offered to enhance the user experience

I agree and disagree Server side validation is to ensure that the format of the data being input to the database, it is not always about the user. To enhance the user experience I would use javascript validation (client side) that could offer hints etc.. Some of the JS libraries (jQuery) offer validation plugins that can go beyond the standard validation error messages. It is a matter of finding the correct balance between the two.

I usually will use javascript validation for the user and a second laer of server side validation as a safeguard and for when js is not on

Of course, you can never trust any user input, be it uploaded in a form or with any other means.

My original concern was about "hard" validation based on reasons that fail to respect the personal background of "soft" human beings.

For security reasons, I would transform anything that might be subject to a mysql statement with htmlspecialchars() or mysql_real_escape_string(). Relying on magic_quotes_gpc isn’t an alternative, because it is deprecated as of PHP 5.3.0 and will be completely removed in php 6.0 an later. Also limiting the number off characters (wherever it is possible) also won’t hurt...

Note: Messing around with user input might have undesirable results. Peter O’Toole wont be happy to see his name "tidied" up to Peter O\’Toole...

My ideas for best practice regarding forms:
1. Suggest changes with client-side javascript AS WELL AS server side php (trust no one...).
2. If changes should be necessary for security reasons, tell the user about the problem and give him the opportunity to edit the questionable content. Also tell him about possible typos or required fields left blank.
3. Give the user the chance to change values until he finally decides to submit the form.
4. Check again for security problems and if safe, continue processing the received data. If user input still not safe, use whatever means necessary to change that.

This would be my basic idea of a "safe" form. Is there anything else, that comes to your minds?

P.S. Has anyone here any experience with suhosin (http://www.suhosin.org/)?