CUTCODEDOWN
Minimalist Semantic Markup

Welcome Guest
Please Login or Register

If you have registered but not recieved your activation e-mail in a reasonable amount of time, please use our Contact Form for assistance. Include both your username and the e-mail you tried to register with.

Author Topic: Deep links not properly redirected from HTTP to HTTPS  (Read 183 times)

fgm

  • Newbie
  • *
  • Posts: 15
  • Karma: +2/-0
    • View Profile
Deep links not properly redirected from HTTP to HTTPS
« on: 23 Dec 2019, 05:48:55 am »
Hi, I've noticed that pages other than index are not properly redirected, giving a 404 error. I've tested it with iridium, firefox and curl.

For example:

http://cutcodedown.com/for_others

« Last Edit: 23 Dec 2019, 05:55:58 am by fgm »

Jason Knight

  • Administrator
  • Full Member
  • *****
  • Posts: 228
  • Karma: +15/-0
    • View Profile
    • CutCodeDown -- Minimalist Semantic Markup
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #1 on: 23 Dec 2019, 08:25:56 am »
Ok, I'm stumped on this one. It's odd, if you include a trailing slash it works, which FF automagically adds if you type it in manually to the address bar, but FF doesn't add it automatically if you follow a link or cut/paste into the address bar...

Works fine in Chrome, Edge, and IE.

Can anyone see what's south with the regex here?

Code: [Select]
RewriteCond %{HTTPS} !on [OR]
RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC]
RewriteRule ^ https://%1%{REQUEST_URI} [L,NE,R=301]

I've used this for near on a decade, never had a problem until about two months ago with FF and crApple products either ignoring redirects and/or mangling the URI's.

Using "!on" instead of "off" seemed to "fix" iOS for about 8 hours...

It's also strange that in addition to the trailing slash, including the www also works.
« Last Edit: 23 Dec 2019, 08:32:04 am by Jason Knight »
"It is amazing what can be accomplished when nobody cares who gets the credit." -- Kelly Johnson

coothead

  • Newbie
  • *
  • Posts: 48
  • Karma: +3/-0
  • I smile benignly
    • View Profile
    • coothead's stuff ~ an eclectic collection
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #2 on: 23 Dec 2019, 09:11:01 am »
Hi there Jason,

I use...
Code: [Select]
RewriteCond %{HTTP_HOST} ^coothead.co.uk$ [NC]
RewriteRule ^(.*)$ https://www.coothead.co.uk/$1 [R=301,L]
RewriteCond %{HTTPS} !^on$
RewriteRule (.*) https://coothead.co.uk/$1 [R,L]

...without any problems.  8)

My test url:-

http://coothead.co.uk/the-pale-blue-dot  ???

coothead
« Last Edit: 23 Dec 2019, 09:16:46 am by coothead »
~ the original bald headed old fart ~

benanamen

  • Global Moderator
  • Newbie
  • *****
  • Posts: 26
  • Karma: +1/-0
    • View Profile
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #3 on: 23 Dec 2019, 01:04:21 pm »
Ok, I'm stumped on this one.

Your close...
(Redirects everything to NON www)

PORT 80 CONF
RewriteEngine On
RewriteCond %{HTTPS} off [OR]
RewriteCond %{HTTP_HOST} ^www\. [NC]
RewriteCond %{HTTP_HOST} ^(?:www\.)?(.+)$ [NC]
RewriteRule ^ https://%1%{REQUEST_URI} [L,NE,R=301]


SSL 443 CONF
Code: [Select]
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.([^.]+\.[^.]+)$ [NC]
RewriteRule ^ https://%1%{REQUEST_URI} [L,R=301,NE]


@coothead, you don't need to hard code domains in the rules.
« Last Edit: 23 Dec 2019, 01:10:33 pm by benanamen »
To save time, let's just assume I am never wrong.

Jason Knight

  • Administrator
  • Full Member
  • *****
  • Posts: 228
  • Karma: +15/-0
    • View Profile
    • CutCodeDown -- Minimalist Semantic Markup
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #4 on: 23 Dec 2019, 05:54:19 pm »
Nope, that's not it. Tried it both direct forced via URI and with the port detection, still nada. Works fine so long as there's a trailing slash. Otherwise it bombs.

I might have to add a 404 redirect to this mess to force it.

-- edit -- even stranger, there's no record of said 404's in any of the error logs on my server.
« Last Edit: 23 Dec 2019, 06:17:16 pm by Jason Knight »
"It is amazing what can be accomplished when nobody cares who gets the credit." -- Kelly Johnson

Jason Knight

  • Administrator
  • Full Member
  • *****
  • Posts: 228
  • Karma: +15/-0
    • View Profile
    • CutCodeDown -- Minimalist Semantic Markup
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #5 on: 23 Dec 2019, 06:01:11 pm »
Actually this is getting stranger and stranger.

that error page is NOT coming from my server. I have custom 404's in place via ISPConfig and they're not being shown.

EVEN stranger? I just added a ErrorDocument line to each and every .htaccess on the server and for that http without a trailing slash it's being ignored.

Without a trailing slash, in Firefox, it is like it's running on raw Apache instead of the proper virtual host. WORSE, a version of Apache that differs from the one my server is running!
"It is amazing what can be accomplished when nobody cares who gets the credit." -- Kelly Johnson

Jason Knight

  • Administrator
  • Full Member
  • *****
  • Posts: 228
  • Karma: +15/-0
    • View Profile
    • CutCodeDown -- Minimalist Semantic Markup
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #6 on: 23 Dec 2019, 06:44:38 pm »
Ok, that's jacked up, for some reason with the slash removed the virtualHosts was refusing to recognize the domain name, so it was falling back on the behavior for calling the IP addy directly.

I've changed my 000-default.conf to include the redirect, and now it seems to work.

Can you folks please verify it's working properly now?

More of a band-aid on a bullet wound fix, need to investigate why the vhosts refuses to recognize the domain name in  the request if a trailing slash isn't present. That's jacked up.
"It is amazing what can be accomplished when nobody cares who gets the credit." -- Kelly Johnson

coothead

  • Newbie
  • *
  • Posts: 48
  • Karma: +3/-0
  • I smile benignly
    • View Profile
    • coothead's stuff ~ an eclectic collection
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #7 on: 24 Dec 2019, 05:40:55 am »
Can you folks please verify it's working properly now?

It now works OK for me. 8)

]coothead
~ the original bald headed old fart ~

ian

  • Keeping on
  • Global Moderator
  • Newbie
  • *****
  • Posts: 28
  • Karma: +0/-0
    • View Profile
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #8 on: 26 Dec 2019, 07:11:44 am »
Fine with:
Ubuntu - Firefox, Opera, Vivaldi and Chromium
Android (9) - Firefox, Samsung Internet and Chrome

But I haven't found any problems in any case?
Our desire for order and definition is often outweighed by our ability.

fgm

  • Newbie
  • *
  • Posts: 15
  • Karma: +2/-0
    • View Profile
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #9 on: 27 Dec 2019, 05:33:04 am »
It works fine now. The program curl is very handy:
Code: [Select]
iron:/home/fgm{3} curl http://www.cutcodedown.com/for_others/
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://cutcodedown.com/for_others/">here</a>.</p>
<hr>
<address>Apache/2.4.25 (Debian) Server at www.cutcodedown.com Port 80</address>
</body></html>

This is how I do the HTTP to HTTPS redirection using Nginx:

Code: [Select]
server {
        listen 80;
        listen [::]:80;
        server_name example.com;
        return 301 https://example.com$request_uri;
}

server {
        listen 80;
        listen [::]:80;
        server_name www.example.com;
        return 301 https://www.example.com$request_uri;
}

server {
        listen 443 ssl;
        listen [::]:443 ssl;
        server_name www.example.com example.com;
        ssl_certificate /etc/ssl/example.com.fullchain.pem;
        ssl_certificate_key /etc/ssl/private/example.com.key;
        # ...
}

HTTPS site works with www and without www, so I avoid a 301 redirect for the first visit. The pages specify the canonical URL with www and use base href.
« Last Edit: 27 Dec 2019, 06:26:49 am by fgm »

Jason Knight

  • Administrator
  • Full Member
  • *****
  • Posts: 228
  • Karma: +15/-0
    • View Profile
    • CutCodeDown -- Minimalist Semantic Markup
Re: Deep links not properly redirected from HTTP to HTTPS
« Reply #10 on: 28 Dec 2019, 06:22:51 am »
It works fine now. The program curl is very handy:
I always find it a bit crippled -- as evidenced by it spitting out the fallback markup that should be ignored, instead of obeying the ACTUAL redirect in the header.

I mean, I get it that curl isn't a UA, but its behavior compared to wget is a bit of a wonk. You end up having to dick around with extra cryptic command line BS to get it to show anything meaningful, and even then it leaves far too much in the callers hands.

I mean what is the option again? -v? -i? --head? Starts reminding me of DrossDOS where you have to type in two lines of flags ending in --please to have the "ls" command not erase the partition.

What was a bad '80's joke is a 21st century reality.

This is how I do the HTTP to HTTPS redirection using Nginx:
Good to know if I ever use nginx, which seems unlikely. I've never had anything I deploy be bloated enough, request heavy enough, or poorly thought out enough for the difference between it and Apache to make any difference.

I mean, I've dealt with it on a few client's servers, but the limitations in what it can do just doesn't seem to justify the MINOR difference in speed... and in cases where it does make a difference -- serving static files -- having a static domain using lightppd blows both out of the water.

More true now that PHP-FPM is commonplace.
« Last Edit: 28 Dec 2019, 10:25:54 am by Jason Knight »
"It is amazing what can be accomplished when nobody cares who gets the credit." -- Kelly Johnson

 

Advertisement