CUTCODEDOWN
Minimalist Semantic Markup

Welcome Guest
Please Login or Register

If you have registered but not recieved your activation e-mail in a reasonable amount of time, or have issues with using the registration form, please use our Contact Form for assistance. Include both your username and the e-mail you tried to register with.

Author Topic: Handling search and User Input?  (Read 2190 times)

GrumpyYoungMan

  • Hero Member
  • *****
  • Posts: 787
  • Karma: +8/-0
    • Grumpy Young Man
Handling search and User Input?
« on: 13 Feb 2020, 02:34:18 pm »
Hello, its me again, the resident forum PIA.

What is the best way to handle a user database search when the results could result in Pagination? is it to send the search string via the URL as a GET request?
Trying to learn a new trick to prove old dogs can learn new ones...

Total Novice have-a go Amateur Programmer - not sure that is the right thing to say... but trying to learn...

Jason Knight

  • Administrator
  • Hero Member
  • *****
  • Posts: 1049
  • Karma: +188/-1
    • CutCodeDown -- Minimalist Semantic Markup
Re: Handling search and User Input?
« Reply #1 on: 13 Feb 2020, 10:17:52 pm »
Not a PIA, users like you asking questions is why forums like this exist!

getData should be avoided for anything relating to security. User logins, administrative actions, etc, etc.

In terms of navigating a site, that's not security related so passing data in the URI is often the best choice. This is often true of things like search so the result can be shared. Even Google passes search strings as "?q=search%20term"

So you should be fine there. So long as the action doesn't modify anything, you're safe to do it that way. To that end, on that table of MOT's you have in your other thread, the user actions like EDIT or DELETE should probably be in a form as buttons so you can pass them in the HTTP headers as postData ($_POST in PHP) instead.

The reason for this is that a malformed link somewhere could trick your system into doing an action like delete. If I made a URI that was "?action=deleteUser&user=1" and that was your trigger for deleting the administrator, it's as simple as tricking you into clicking on that in something like a forums to delete your admin account... That's the core of XSS exploits and why GET should only be used for non-secure actions users might want to link to.

Learning when to use GET and when to use POST is one of the many hurdles in learning this stuff.
We are all, we are all, we are all FRIENDS! For today we're all brothers, tonight we're all friends. Our moment of peace in a war that never ends.

John_Betong

  • Full Member
  • ***
  • Posts: 218
  • Karma: +24/-1
    • The Fastest Joke Site On The Web
Re: Handling search and User Input?
« Reply #2 on: 13 Feb 2020, 11:32:48 pm »
@GrumpyYoungMan
> What is the best way to handle a user database search when the results could result in Pagination?

I use AJAX for better or worse and curious to know if there are security implications?

https://dasabookcafe.tk
Retired in the City of Angels where the weather suits my clothes

Jason Knight

  • Administrator
  • Hero Member
  • *****
  • Posts: 1049
  • Karma: +188/-1
    • CutCodeDown -- Minimalist Semantic Markup
Re: Handling search and User Input?
« Reply #3 on: 14 Feb 2020, 02:05:31 am »
I use AJAX for better or worse
Worse if you don't have scripting off graceful degradation... AJAX should be an enhancement, not your only means of functionality.

But in general that page is a mess with the inlined images, strange nonsensical use of EM and STRONG, non-breaking spaces, SVG wrapping a non-SVG image for Christmas only knows what, gibberish cryptic classes only you would ever know what they mean, endless pointless classes for nothing, breaks and rules where they make no structural sense, double-breaks doing margin or padding's job, paragraphs around non-paragraph content, static scripting in the markup, nested DL where NONE of it is terms and definitions, broken redundant favicon LINK tags, outdated and outmoded type attributes... It's got problems.

Though I'm not saying don't use AJAX, just make it work without AJAX first!
« Last Edit: 14 Feb 2020, 02:12:00 am by Jason Knight »
We are all, we are all, we are all FRIENDS! For today we're all brothers, tonight we're all friends. Our moment of peace in a war that never ends.

GrumpyYoungMan

  • Hero Member
  • *****
  • Posts: 787
  • Karma: +8/-0
    • Grumpy Young Man
Re: Handling search and User Input?
« Reply #4 on: 14 Feb 2020, 10:35:36 am »
Not a PIA, users like you asking questions is why forums like this exist!
Not sure, but sometimes it feels like I am spamming your forums...


getData should be avoided for anything relating to security. User logins, administrative actions, etc, etc.

In terms of navigating a site, that's not security related so passing data in the URI is often the best choice. This is often true of things like search so the result can be shared. Even Google passes search strings as "?q=search%20term"

So you should be fine there. So long as the action doesn't modify anything, you're safe to do it that way. To that end, on that table of MOT's you have in your other thread, the user actions like EDIT or DELETE should probably be in a form as buttons so you can pass them in the HTTP headers as postData ($_POST in PHP) instead.

The reason for this is that a malformed link somewhere could trick your system into doing an action like delete. If I made a URI that was "?action=deleteUser&user=1" and that was your trigger for deleting the administrator, it's as simple as tricking you into clicking on that in something like a forums to delete your admin account... That's the core of XSS exploits and why GET should only be used for non-secure actions users might want to link to.

Learning when to use GET and when to use POST is one of the many hurdles in learning this stuff.
I think I will lean more towards POST for everything I can - unless it has to be sent via GET...

Thanks for continued feedback and support... I appreciate it...
Trying to learn a new trick to prove old dogs can learn new ones...

Total Novice have-a go Amateur Programmer - not sure that is the right thing to say... but trying to learn...

John_Betong

  • Full Member
  • ***
  • Posts: 218
  • Karma: +24/-1
    • The Fastest Joke Site On The Web
Re: Handling search and User Input?
« Reply #5 on: 14 Feb 2020, 11:56:56 am »
I use AJAX for better or worse
Worse if you don't have scripting off graceful degradation... AJAX should be an enhancement, not your only means of functionality.

But in general that page is a mess with the inlined images, strange nonsensical use of EM and STRONG, non-breaking spaces, SVG wrapping a non-SVG image for Christmas only knows what, gibberish cryptic classes only you would ever know what they mean, endless pointless classes for nothing, breaks and rules where they make no structural sense, double-breaks doing margin or padding's job, paragraphs around non-paragraph content, static scripting in the markup, nested DL where NONE of it is terms and definitions, broken redundant favicon LINK tags, outdated and outmoded type attributes... It's got problems.

Though I'm not saying don't use AJAX, just make it work without AJAX first!
The VPS site has recently had big problems with 100% CPU Usage necessitating a reboot. I have been firefighting trying to keep the site alive and have not been able to pinpoint the problem. It could be Ubuntu updates conflicting with my script or corrupt imported XLS spreadsheets. 

The site has been running for a couple of years and Cron jobs used to extract XLS spreadsheet links from three web pages. CSV data extracted and uploaded to PHP MySql tables which are validated before making available to the search routine.

A week ago I added a Google Books API and exceeded the 1,000 daily quota in only a couple of hours. Google display an 'ORRIBLE quota exceeded message so removed the new search link and left the empty DIV. I'm still waiting for credit card approval to increase the quota. I am not alone...
https://www.reddit.com/r/google/comments/8b7k0k/help_needed_google_books_api_quota/
« Last Edit: 14 Feb 2020, 11:59:41 am by John_Betong »
Retired in the City of Angels where the weather suits my clothes

Jason Knight

  • Administrator
  • Hero Member
  • *****
  • Posts: 1049
  • Karma: +188/-1
    • CutCodeDown -- Minimalist Semantic Markup
Re: Handling search and User Input?
« Reply #6 on: 14 Feb 2020, 03:28:18 pm »
The VPS site has recently had big problems with 100% CPU Usage necessitating a reboot. I have been firefighting trying to keep the site alive and have not been able to pinpoint the problem. It could be Ubuntu updates conflicting with my script or corrupt imported XLS spreadsheets. 
Hmm... some things to check.

1) Are you hemorrhaging PHP errors silently? If in top that usage is mostly IOWAIT it could be you're overflowing your logs and the system is hogging time whenever it has to compress and back them up. This bit a client of mine multiple times as their in-house guys kept screwing over their ancient copy of UBBThreads (ick) with "security code" that filled the logs to the "compress now" threshold every 8 hours, with logs so big they took an hour to compress them into .gz. Naturally after a couple days the machine started running out of disk space too. Eventually it was so utterly banjaxed I said screw it, wiped their forums, and moved them to SMF which didn't need all the bizzaroland patching they were trying to do (against my advice).

2) Excessive HTML -- particularly if you're doing realtime base64 of that image -- can cause a lot of server load.

3) Are you 100% sure you've got all the right fields indexed?

4) make sure those cron jobs are NOT all running the same time of day.

5) Could it be a RAM bottleneck? If going fulltext search I suggest at LEAST 2gb of RAM on your VPS plan.

6) Are you on MariaDB instead of mySQL? Oracle seems to be intentionally neutering mySQL by making it suck more CPU and memory than it used to for some vague "oh it'll be faster" -- no, no it isn't. Whilst MariaDB is a drop-in replacement fork that came about because the world doesn't trust Oracle not to screw it up (see OpenOffice vs. LibreOffice), the two are really starting to diverge from each-other now on the back-end.

One final thing you might try is a optimized fulltext engine like Sphinx:

http://sphinxsearch.com/

I've added it to forum software before and it made a huge difference in performance both in terms of time/cpu use and looking for specific text. Never programmed for it directly, but it could be a possible solution to your load issues. It sits on top of / between your SQL engine and the server-side language making it fairly simple (from what I read in the docs) to add/implement.
« Last Edit: 14 Feb 2020, 03:33:31 pm by Jason Knight »
We are all, we are all, we are all FRIENDS! For today we're all brothers, tonight we're all friends. Our moment of peace in a war that never ends.

GrumpyYoungMan

  • Hero Member
  • *****
  • Posts: 787
  • Karma: +8/-0
    • Grumpy Young Man
Re: Handling search and User Input?
« Reply #7 on: 15 Feb 2020, 02:19:45 am »
I’m no expert but I’m with Jason - if the site is using old PHP code and logging errors caused by Deprecated PHP features maybe?
Trying to learn a new trick to prove old dogs can learn new ones...

Total Novice have-a go Amateur Programmer - not sure that is the right thing to say... but trying to learn...

John_Betong

  • Full Member
  • ***
  • Posts: 218
  • Karma: +24/-1
    • The Fastest Joke Site On The Web
Re: Handling search and User Input?
« Reply #8 on: 9 Sep 2020, 02:10:41 am »
I was searching for a recent topic and unable to find it but did find this unanswered post...

Just a quick note to say that I rewrote the offending update script and liberally added PHP  flush();  in umpteen places. The revamped  script now completes in about fifteen seconds!  It originally took over a minute!

Not bad for converting three XLS Spreadsheets with over 30,000 records into two MySQLi database tables using an inexpensive VPS with only one Mb  :)
Retired in the City of Angels where the weather suits my clothes

 

SMF spam blocked by CleanTalk

Advertisement