CUTCODEDOWN
Minimalist Semantic Markup

Welcome Guest
Please Login or Register

If you have registered but not recieved your activation e-mail in a reasonable amount of time, or have issues with using the registration form, please use our Contact Form for assistance. Include both your username and the e-mail you tried to register with.

Author Topic: Squire Rethink  (Read 879 times)

benanamen

  • Full Member
  • ***
  • Posts: 149
  • Karma: +12/-0
Squire Rethink
« on: 26 Nov 2020, 02:10:28 pm »
A few days ago I claimed that Squire 3 was overly complicated for outputting static pages. I have since created the exact same site with a whole new architecture that is so simple in comparison it will blow your mind.

You can do file compares of the browser source to see the output is the same as Squire 3. I will give everyone a few days to try and break it and then I will post a repo of the code.

Let me know if you find a problem.

Here are some stats

Squire 3
16 Folders - 41 Files

Squire Rethink

6 Folders - 28 Files

https://galaxyinternet.us/demos/squire-rethink
To save time, let's just assume I am never wrong.

coothead

  • Full Member
  • ***
  • Posts: 192
  • Karma: +50/-0
  • I smile benignly
    • coothead's stuff ~ an eclectic collection
Re: Squire Rethink
« Reply #1 on: 26 Nov 2020, 02:32:50 pm »
Hi there benanamen,

I cannot comment about Squire 3, it is
way above my head, but you did say...

Quote
Let me know if you find a problem.

View Source shows this...

Code: [Select]
demos/squire-rethink<!DOCTYPE html><html lang="en"><head><meta charset="utf-8">

...which may or may not be a problem
that you wish to address.  :)

coothead
~ the original bald headed old fart ~

GrumpyYoungMan

  • Hero Member
  • *****
  • Posts: 519
  • Karma: +5/-0
    • GrumpyYoungMan
Re: Squire Rethink
« Reply #2 on: 26 Nov 2020, 02:34:33 pm »
Sometimes people do have a tendency to over think and complicate things... and maybe over look the simple approach and that is also based on your own perspective...

I think we are all guilty of that... not that I am saying Jason has done that...

I will look forward to reviewing your code...
Trying to learn a new trick to prove old dogs can learn new ones...

Total Novice have-a go Amateur Programmer - not sure that is the right thing to say... but trying to learn...

benanamen

  • Full Member
  • ***
  • Posts: 149
  • Karma: +12/-0
Re: Squire Rethink
« Reply #3 on: 26 Nov 2020, 02:49:24 pm »
Thanks. It was a left over debug echo. I was hurrying to get it up before I had to leave for Thanksgiving dinner.

Any comments I make on others code is not an attack on the person. It is strictly about code and becoming better programmers.

After digging into Squire 3 there are a few techniques I will be adopting as part of my standard repertoire. Much of what I do now are things I have learned from other people over the years. So thanks to Jason for already teaching me some new things with Squire 3.
« Last Edit: 26 Nov 2020, 02:55:16 pm by benanamen »
To save time, let's just assume I am never wrong.

GrumpyYoungMan

  • Hero Member
  • *****
  • Posts: 519
  • Karma: +5/-0
    • GrumpyYoungMan
Re: Squire Rethink
« Reply #4 on: 26 Nov 2020, 02:59:45 pm »
Any comments I make on others code is not an attack on the person. It is strictly about code and becoming better programmers.
Everyday is a school day! And I agree, feedback should never be taken personally - if it was I would be long gone - as I do not belong in this world...
Trying to learn a new trick to prove old dogs can learn new ones...

Total Novice have-a go Amateur Programmer - not sure that is the right thing to say... but trying to learn...

GrumpyYoungMan

  • Hero Member
  • *****
  • Posts: 519
  • Karma: +5/-0
    • GrumpyYoungMan
Re: Squire Rethink
« Reply #5 on: 26 Nov 2020, 03:01:30 pm »
Thanks. It was a left over debug echo. I was hurrying to get it up before I had to leave for Thanksgiving dinner.

Any comments I make on others code is not an attack on the person. It is strictly about code and becoming better programmers.

After digging into Squire 3 there are a few techniques I will be adopting as part of my standard repertoire. Much of what I do now are things I have learned from other people over the years. So thanks to Jason for already teaching me some new things with Squire 3.
Oh happy Thanks Giving!
Trying to learn a new trick to prove old dogs can learn new ones...

Total Novice have-a go Amateur Programmer - not sure that is the right thing to say... but trying to learn...

John_Betong

  • Full Member
  • ***
  • Posts: 156
  • Karma: +14/-1
    • The Fastest Joke Site On The Web
Re: Squire Rethink
« Reply #6 on: 26 Nov 2020, 10:31:16 pm »
@benanamen,
> Let me know if you find a problem.

Invalid "Three Stooges" image links on the Test page?

>  A few days ago I claimed that Squire 3 was overly complicated for outputting static pages. I have since created the exact same site with a whole new architecture that is so simple in comparison it will blow your mind.

I took the opposite approach:

1. tried to simplify the project to make it easier to update

2. moved the CSS, image and JavaScript files to the common public_html/assets folder

3. separated the "system" and "application" files into folders that can be referenced even if the folder is above the root




I have since removed the "ZZZ-ORIGINAL=SOURCE-CODE" folder.

Is there an application that counts the folders and files or should I do that manually?


« Last Edit: 26 Nov 2020, 10:55:14 pm by John_Betong »
Retired in the City of Angels where the weather suits my clothes

benanamen

  • Full Member
  • ***
  • Posts: 149
  • Karma: +12/-0
Re: Squire Rethink
« Reply #7 on: 27 Nov 2020, 02:27:16 am »
Quote
Invalid "Three Stooges" image links on the Test page?

Thanks John. Fixed. I will have time tomorrow to fine tune without being on a time crunch.

Quote
1. tried to simplify the project to make it easier to update

I did the same thing by redoing the entire architecture. I will be interested to hear everyone's feedback.

Quote
Is there an application that counts the folders and files or should I do that manually?

If you are on windows right click the project folder and select Properties. If you have other files in there such as .git you will need to open the project folder, select only project files and folder, then do right click->Properties

By the way, how did you make the Dir Tree image?
To save time, let's just assume I am never wrong.

Jason Knight

  • Administrator
  • Hero Member
  • *****
  • Posts: 694
  • Karma: +128/-1
    • CutCodeDown -- Minimalist Semantic Markup
Re: Squire Rethink
« Reply #8 on: 27 Nov 2020, 02:52:20 am »
I will give everyone a few days to try and break it and then I will post a repo of the code.
More interested in that code side, I'm wondering if you yanked a bunch of bad practices and/or groundwork for expansion/scalability. A lot of the things you might object to or remove could have been in there for a reason... like the internal scope isolation to make many common code elevation attacks fall flat on their face... or not actually having code in the static configuration file. (so many people use .php files for that), etc, etc.

Once I get caught up on dishes from Thanksgiving, I've got the whole weekend free for once, so coding will start in earnest on getting the database and installer working, then to dive into an admin panel to manually add/remove users. Then if there's time I might tackle registration as well.

Assuming nothing my life goes pear shaped.

-- edit -- in particular given the drop in file count, I'd be concerned about you undoing DCR.
« Last Edit: 27 Nov 2020, 05:45:10 am by Jason Knight »
I'll fix every flaw, I'll break every law, I'll tear up the rulebook if that's what it takes. You will see, I will crush this cold machine.

John_Betong

  • Full Member
  • ***
  • Posts: 156
  • Karma: +14/-1
    • The Fastest Joke Site On The Web
Re: Squire Rethink
« Reply #9 on: 27 Nov 2020, 04:00:53 am »
@benanamen,
> If you are on windows right click the project folder and select Properties... select only project files and folder, then do right click->Properties

I’m running Linux :(

>By the way, how did you make the Dir Tree image?

Sublime editor has a tree view of open files and Linux has an application to screen dump or cut a portion of the screen. I used the latter.
Retired in the City of Angels where the weather suits my clothes

benanamen

  • Full Member
  • ***
  • Posts: 149
  • Karma: +12/-0
Re: Squire Rethink
« Reply #10 on: 27 Nov 2020, 01:39:24 pm »
Thanks John. I see I can screenshot my IDE panel and get the tree image I want.
Quote
More interested in that code side

Sure. I would ask if your going to claim that my changes make the code vulnerable to code elevations or what ever you may claim is a problem that you also provide a demonstration of of the flaw. It's easy to just make accusations that's why I have given an open invitation to break/hack the demo I put up.

You have already wisely perceived some things I changed so you should not be surprised. If you say there is a problem then just demonstrate it like John and coothead did.

The major point of my fork of Squire 3 is the change in Architecture. I have intentionally left out the gzip code which can be a topic of its own and a couple small niceties like dynamic title generation as it isn't relative to my point of unnecessary architecture complexity and is easily implemented if someone wants. Since I have time, I may just end up doing an exact match of browser source just to keep output the same exact as yours. (I think it already is except <title></title>)

Quote
A lot of the things you might object to or remove could have been in there for a reason

If that is the case then please just demonstrate how it is a problem that I left it out. I know there was some dead code that isn't used or "reasons" not yet implemented but could/would be once you have completed the build which is why I was wanting to hold out on comments or refactoring until you were done. Since this is moving slowly I just went with it as is and focused on the code just for static page output. (Pretty sure it is already capable of dynamic content as well)

Never heard of DCR. What is it?

Edit: Fixed Typo
« Last Edit: 27 Nov 2020, 10:50:43 pm by benanamen »
To save time, let's just assume I am never wrong.

Jason Knight

  • Administrator
  • Hero Member
  • *****
  • Posts: 694
  • Karma: +128/-1
    • CutCodeDown -- Minimalist Semantic Markup
Re: Squire Rethink
« Reply #11 on: 27 Nov 2020, 04:44:09 pm »
Never heard of DCR. What is it?
Dead Code Reduction.

In compiled languages unused code isn't included in the executable if the compiler is worth a damn, but in interpreted languages we have no such auto-optimization... but even in compiled languages where that exists, there are times when the executable is called that the whole program has to be loaded, even if it isn't used that time the user runs it. This isn't generally a problem for client side apps, but with server stuff well, there's a reason we tend to run them as services.

In source-delivered interpreted languages though, we take a huge hit for unused branches in our code. We need the branches there, but loading a branch unused is a waste.

Thus most of what goes into my squire.lib.php either dynamically loads content as needed using smaller wrappers -- see Lang:: and Bomb:: -- or otherwise only loads what the Request needs.

It's a problem I see in a lot of heavily "object based" code, is that they are  often less code in total, but they run more code every blasted time they start because EVERYTHING a class does is loaded, whether it's used or not. Whilst that's oft unavoidable on classes like Database, It's a disaster when done with templates.

Like I know a lot of people would say to include /template/default/forms.template.php ... but what if we're building a system that doesn't have a form? Or take the extras sections as an example. Should we load its template methods even if we're not using them?

We get into user control, are we really going to load everything you can do to a user -- even when the separations should be there for security (like not allowing "User" object to do anything but the "Read" and "Update" parts of CRUD by crapping it all into one giant object, for commands only the admin should have access to?

Languages though is the best example. For the most part if a "section" of our languages is used there are likely some values to be loaded, but that doesn't mean we should waste time -- and system memory -- loading every Joe-blasted regionalization string every blasted page-load. Which I've seen people do as well.. Just because I need Lang::get('@hashFailure_login') doesn't mean I need to load every single login error's content text. Which is why one of the changes in the upcoming code is to break all the different HTTP error codes out into their own separate /lang/language/httpErrors/###.txt files.

Smaller less frequently used files can also lead to memory reductions that play better with caching models at the filesystem cache and PHP opcode cache levels.

Hence a DCR pass in languages like PHP where the programs don't have to be monolithic oft involves figuring out what's used on every pass, and what is not. It means more includes and more separate loads, but it can reduce the memory footprint in a way that results in better/faster caching and faster execution times.

Whilst writing the functionality first pass it's often easy to forget this, hence it helps to 1) build the architecture with this in mind, and 2) make a few passes through the whole code to think about what's used every page load, what's used frequently, and what you need to scream at yourself "Oi, leave it out gimboid!"

I've added a few things to the to-do list for this after the base admin panel and the CRUD for users is implemented. (hoping to start that tonight).

In no particular order:

template stacks -- a master Template object where the template functions are registered as anonymous as multiple arrays of function. This will allow extensions/modules to hook themselves into the template stream wherever we leave "hooks" for it. "printf" style strings may also be supported for the people who dislike working with functions for everything, though I prefer to "keep it in my pants" on that with non-content stuff given the gross inefficiencies of complex string processing.

I'll provide it, but likely will use it minimally "out of box" for the same reason I prefer comma delimits, and think that high-string processing overhead templating in PHP --- smarty/handlebars/whatever -- or even just double-quoted strings with variables {} in them is mind-numbingly dumbass trash I would rarely if ever allow in my code. I'm sure there's an exception somewhere, but...

process stacks -- I've already implemented this for just "extras" but it should be expanded to make adding third party mods easier. @benanamen you'll be happy to know extras is not opt-in by setting them with Extras::add, instead of being opt-out with Settings::set

Dynamic user data -- another table for users with a many to one of data the admin will be able to choose/add/customize. The typical id/user_id/name/value table style.

profile modal -- rather than completely pollute the main menu, move admin controls, moderation controls, and other such user actions to their own modal.

Right now though I'm working on my milestone copier, which is about as close to version control as I ever get since again, I find most version control junk. I'd rather have people send me date-coded files of their changes, than waste my life dicking round with asshat broken slow bullshit like GIT. Again I'd sooner eat a bullet than deal with GIT or anything like it given the headaches, counter-intuitive interface, and just plain rubbish they seem to be.

I can hand assemble RCA 1802 machine language, but I can't wrap my head around the benefits of GIT and its kine. I think it's mix of issues.

1) disgust from how I've watched so many "project managers" do everything EXCEPT manage projects by using Github so they can spew bullshit bingo in meetings and then play Farmville all day instead of doing actual work.

2) A mental block much akin to how I can't understand/do "visual programming". I can't use RAD's either, gets in my damned way too much and doesn't match my workflow.

3) I've watched it screw codebases because people are lazy in commenting their commits, at which point why the hell bother. At least if you have an ACTUAL FLIPPING PERSON reviewing changes, approving and merging new code with the responsibility of checking both intent and efficiency, you have a chain of both liability and can keep a project on track. 

But no, apparently ^C^V into a new .txt with a timestamp in the filename with text describing the how/what/why of the changes, and having an actual person riding herd on the development team -- aka MANAGING THE FLIPPING PROJECT -- is "too hard" these days.

That said, I am very interested to see what you did to reduce the code size so much. I'm all about smaller code, but not at the cost of memory, performance, or code clarity. Though those three are often akin to
the old joke about "soon", "cheap", and "well written"; pick only two.
I'll fix every flaw, I'll break every law, I'll tear up the rulebook if that's what it takes. You will see, I will crush this cold machine.

Jason Knight

  • Administrator
  • Hero Member
  • *****
  • Posts: 694
  • Karma: +128/-1
    • CutCodeDown -- Minimalist Semantic Markup
Re: Squire Rethink
« Reply #12 on: 27 Nov 2020, 05:52:15 pm »
Oh, and as to code elevation attacks, we don't have anything that could open that hole YET, so there's nothing to really go after. The Request method is (theoretically) checking for up-tree hacks, the .htaccess is preventing direct calls to sub-libraries... There's no working forms or database to exploit...

And even if I were to I don't have access to YOUR wireless to slap aside WAP and exploit your session, I'm not on a router/switch up-stream of you so MITM is right out, etc, etc.

But more than that this code is about layout the groundwork for protection mechanisms. Sloppy PHP loves to let hackers in to run arbitrary code... and this isn't about our PHP no matter how good we write it, it's about what the next slob to come along does with it.

Let's get a whip for Toby and use turdpress as an example. Back in the '00's WP itself was horrifyingly insecure, it won the 2008 pwnie for M4ss 0wnage after all, and REALLY should have won again for 2017!

Ever since version 3 that's actually no longer true if you run it out of box standalone. They made a number of changes that if you do not use mods/extensions/plugins/whatever word for it is hot and trendy this week did a good job of shoring up that outer wall.

But the moment you add third party code, all bets are off. If you look at the code elevation and other hacks in WP's CVE:

https://www.cvedetails.com/product/4096/Wordpress-Wordpress.html?vendor_id=2337

You start going through the listings you'll see a lot of "may require third party plugin" or "specific to plugin xxx" or some such across the whole mess. It's no longer their code, but that of what people add to it.

But a LOT of these are based on actual issues under the hood with WP because they only have "one ring" security. They have an outer wall, but that outer wall has multiple entry vectors thanks to the sheer number of files you can blindly call that expose information. They store the database CREDENTIALS in "define" which is akin to leaving the keys to the kingdom hanging on a wall in the town square; someone gets in the wall it's exposed.

Around two-thirds the exploits present there could be avoided by:

1) not using the URI to say what you're doing the CUD in your CRUD on. That's POST's job.

2) not having the SQL credentials in the super-global namespace. (define)

3) not having the database connection accessible in places it shouldn't even exist like the output handler side of things.

4) restricting the system so only pre-declared queries can be used.

5) Not allowing multiple entry vectors or blindly doing include() without scope restriction. Control your scope bleed!

The other third require changes to PHP itself, such as how I'm always talking about how dumbass it is that you're allowed to fopen/readfile/get_file_contents .php source files. It leaves PHP "insecure by design". There NEEDS to be a mechanism to restrict file access to specific data (like SQL credentials) and certain bits of live source. Being able to write to .php files from PHP is just dumbass insecure trash, and being able to read them isn't much better!

In a lot of ways though it's not about if we're vulnerable or not now, it's about reducing what a cracker can do if they manage to get in. Protections and good practices that can save us a slew of headaches later. If you provide those protections from the ground up, it's harder for later coders to screw things up.

Multiple-ring protection, like REAL castles. And not the phony pretend 19th century "manor houses" that people who know jack about shit call "castles". You have the perimeter wall, the outer promenade, one or more inner rings of walls with courtyards, and then the inner keep which itself is designed around choke points.

This "ground up methodology" a lot of people will quote Knuth out of context on, but it's not premature optimization! It's laying a foundation and providing protections. Akin to Pascal's "the compiler won't let you shoot yourself in the foot" or Ada's "F*** the second amendment; we're taking that gun away from you and melting it down.".

Gah, if only Ada were practical for web development. But no, that's "too hard for the normies".

Anyhow, neither of us have done enough with this to have real vulnerabilities yet; that doesn't mean you don't start out with those protections at the architectural level. Think of it as over-provisioning, a sound engineering practice. As Scotty would say, a good engineer is always a wee bit conservative, at least on paper. When designing a structure you figure out the maximum load/capacity it "needs", and then you double that for the actual design.

Pisses off the pencil-pushers, but makes things safer and provides liability protection.

I cannot provide examples based on what we've done so far, but you can go browse cvedetails for hundreds of examples of the type of stuff I'm trying to prevent. Though again, PHP itself kind-of pisses on these efforts. One file_get_contents and game-over.
« Last Edit: 27 Nov 2020, 06:07:15 pm by Jason Knight »
I'll fix every flaw, I'll break every law, I'll tear up the rulebook if that's what it takes. You will see, I will crush this cold machine.

benanamen

  • Full Member
  • ***
  • Posts: 149
  • Karma: +12/-0
Re: Squire Rethink
« Reply #13 on: 27 Nov 2020, 08:19:08 pm »
There's no working forms or database to exploit...

Yet another reason I was hoping to wait until you were farther along before I really jumped in. Sounds like you are getting pretty close though. The DB part is what I have been wanting to see from the beginning. Perhaps we can soon return to the "To Route Or Not To Route" discussion.

A couple points about the "Squire Rethink" version is that it is a refactor of Squire 3 Architecture and not how I would normally do mine, although it is similar in some respects.

I have kept the entire code base in a single folder that runs as the server root or any number of sub-directory nesting with one simple hard coded config. I think Squire 3 does it dynamically. There is nothing above root as I would normally do.

I know there are hosts that do not let you go above root. I feel you need to get a new host if that is the case. You shouldn't have to code the app to make up for poor hosting. Also part of why I left out the Gzip code. That belongs in the server config or the vhost. Same goes for X-Frames Header.

There is also no Router as they are commonly known to be in keeping with Squire 3.

I have intentionally left out comments and explanations as I believe the code is simple and intuitive enough that anyone with basic knowledge can easily see how it works and how to add pages. I also want to hear feedback on that without having provided any assistance on how things work.

For now I am intentionally leaving out any code for dynamic content since we don't have any yet in Squire 3 except for setting Titles, Keywords and "Extras and the like". As I said, the point of the rethink is about the Architecture change.

Edit: Fixed Typo
« Last Edit: 27 Nov 2020, 11:12:24 pm by benanamen »
To save time, let's just assume I am never wrong.

benanamen

  • Full Member
  • ***
  • Posts: 149
  • Karma: +12/-0
Re: Squire Rethink
« Reply #14 on: 27 Nov 2020, 08:59:31 pm »
To help people break "Squire Rethink" here is the exact structure. I will post the code soon.

To save time, let's just assume I am never wrong.

 

SMF spam blocked by CleanTalk

Advertisement