Making Webserver Available Across Internet
Raspberry Pi edition... with much for other readers
filename: pt0FirstWebserver2.htm
Welcome!
Pre-requisites...
This is part 2 of some information showing you how to use a Raspberry Pi for a webserver. The first part, Webserver1 explained how to create a webserver on a Pi. This part explains how to make an existing webserver accessible across the whole internet. It is written for Pi users, but will have many elements that apply regardless of what webserver you are trying to make available.
(At the end of this, you are invited to go on to a third page, which discusses the benefits of adding .php service to your webpage, and shows you how to do that for a Raspberry Pi. )
Creating the webserver is a simple matter. When I'd been using a Pi for less than a week I created one in about 25 minutes, including writing the process up as I went along.
Connecting it to "the outside world" is where the hair tearing may begin.
Before you try to connect your webserver to the outside world, get it working on the PC the webserver is on. That is covered in Webserver1, to which there is a link just above here.
If, on that machine, you put "localhost" (without the quotes) into an internet browser (which I often just call "a browser"), you should see a page from the server. You should be able to set up a page with a name like MyWebPage.htm, and "localhost/MyWebPage.htm" (on the machine the webserver is on) should bring up that page.
It is okay, and not a big deal, if at this stage your webserver is fussy, and won't serve ThisPage.htm if you ask for thispage.htm, i.e. is sensitive to the case of letters in the page request. That can be fixed.
Okay so far? Good! Onward!...
Previously, we accessed the webserver from the machine it was on (how hard can that be?), or from another computer on the same lan, using the webserving computer's "hostname".
Neither will be possible for accessing the machine from outside the LAN the webserver is on.
By the way... for testing you can access the webserver as if you are outside the LAN even from a computer on the same LAN as the webserver.
How it works...
Let me go into that a bit further.
At the moment, you are reading...
http://sheepdogguides.com/pi/pt0FirstWebserver2.htm
You either put that into your browser's address bar, or used a search engine to find it, and the browser "put that into the address bar".
The first part, everything up to the first single "/"...
http://sheepdogguides.com/
.. said "make an http-type request for something from the webserver "at" sheepdogguides.com."
"SheepdogGuides.com" is a domain name that I own and control.
The rest, the...
pi/pt0FirstWebserver2.htm
... only worked because I have a "pi" folder in the folder of things served by my server, and a file called "pt0FirstWebserver2.htm" in that. That file holds the html that results in what you are reading.
That's "all" there is to it!
Please re-read the previous section.
"All" you need to do is to fix things so that you own a domain name, and then make that take people to your webserver.
Neither part is trivial, but I will take you through the steps. First a little tutorial on IP addresses.
IP addresses... LIPAs and WIPAs
Most people go happily about their lives without ever needing to know about IP addresses. (Aren't you glad you took on running your own webserver? Do be! It will be worth it!)
Devices on LANs and on the internet (WAN) are, when you look into the details usually (thankfully) hidden from us, have numeric addresses. If you are "just" connecting to something across a LAN, you will use it's LAN IP address (LIPA). If you are connecting to something across the internet, you will use it's Wide area network IP Address (WIPA). (What does "IP" stand for? "Internet Protocol".)
Again. Please. Go back, re-read that recent bit. It isn't so very hard, but there are things that are easily muddled.
Now... deep breath... Part 1
Here's where things get tough. I'm going to go through some stuff in a "theoretical" way in Part 1. I will then go through them again, with the details of How You Do It in part 2.
Here begins "Part 1"... the theory...
Suppose you go to a webserver that I am running. I have the hardware here beside me. When you ask for...
http://ThatServer.com/MainPage.htm
... a great deal happens.
(This example assumes that I have licensed the domain name "ThatServer.com", and set up a DDNS service. (I'll explain that in a later section.)
The request goes out from your computer, onto the internet.
First, a DNS server is consulted. (That's different from the DDNS thing.) It is a bit like a phone book. It takes "ThatServer.com" as input, and returns a WIPA. Something like 34.210.42.14. We, as users, don't know, don't care. "The system" takes care of that. Which is just as well, because ThatServer.com's probably changes from time to time, but the DNS service will know about it, if it does.
Then the request is sent to the device at the WIPA that turned out (courtesy of the DNS server) to be the right one for ThatServer.com.
That device will be the router that connects your LAN to the internet. As far as "the outside world" of the internet is concerned, everything you do, with any device on your LAN (including things on it "merely" by WiFi) happens "through" that one WIPA.
That's why your router is really clever.
Suppose one family member ("L") is using a laptop in the kitchen to watch a YouTube, and another family member ("T") is in the den, reading this document on a tablet. L's request went out to YouTube, T's requests went out to SheepdogGuides. "Stuff" was sent back... all from and to the one WIPA, remember. Inside the router, everything was taken care of so that the pages L requested went to L, the pages T requested went to T.
Cool!
The trick of it is that each device connected to the LAN has a LAN IP address (LIPA). It looks very like a WIPA.
L's laptop talks to the router over the laptop's LIPA. The router, on L's behalf sends requests out, over the WIPA, to the internet. Those requests "carry with them" TWO return addresses, two IP addresses: The WIPA and the LIPA. They get returned as part of the response. One return address to get the response to the router, and the second so that the router knows where to pass the response.
If we attach a webserver to the LAN, it will have to go at a unique LIPA... unique to that LAN. As LIPAs usually start 192.168.0. there must be an awful lot of devices "at" 192.168.0.10 in the world. As long as no two of them are on the same LAN, that's okay.
Our router, as I said, will have a WIPA. We have to arrange for the DNS servers to know that THAT WIPA "goes with" our domain name. ("ThatServer.com", in my example.)
Deep breath... Part 2
So that's the theory of LIPAs and WIPAs, with DNS thrown in.
Now we'll turn to how you make it happen, in practice.
Almost all of what follows applies whether you are putting a webserver in a Pi on the internet, or, say, an IP cam. They are general techniques. Once you have put a webserver on the internet, "doing it again", to, say, put am IP cam on it too is a dawdle, because you've already done almost everything you need to.
Remember (see top of page): This assumes you have a "working" webserver on your LAN. You can fetch pages from it by saying "go to [hostname], fetch [page]".
Right. All on same page. Good.
First we need to know "where" the webserver is on the LAN. This is worth several pages on it's own, which I will try one day to write. Pi's seem averse to being given static LIPAs. I find this puzzling and annoying. I finally managed to get my Pi webserver "at" an unchanging LIPA: Thankfully, the DHCP inside my router is able to "reserve" a LIPA for a device based on the device's MAC ((to be explained)). Beware: The ethernet (hard wired) interface to your Pi and the WiFi interface to your Pi have different MACs.
On the Pi, under the CLI, issue an ifconfig command. (The second letter is an F, notice. Not the P you have on a Windows machine.)
In the resulting slew of information (if, as I would advise, at least to start off) you are going to put the Pi on the LAN with a cable, not WiFi, look in the "obal" section. There should be an entry after "ether". It should be 6 pairs of characters, separated by colons. The characters can be 0-9, a-f, e.g...
ether b8:27:eb:53:3a:fc
This is the machine's MAC. (Don't fall for the MAC associated with the WiFi interface)
Tell your router (if you can) to always assign the LIPA of your choice (from the DHCP pool... you may have put some LIPAs outside that pool, when doing other things.) KEEP CAREFUL RECORDS!
Let's say you put your Pi on 192.168.0.180
Once you've (successfully!) done that, if, earlier (as explained in part 1 of this guide) you were using..
http://pi5aug18/MyFirstPiWebpage.htm
.. to access that page on your Pi webserver (having given the Pi hostname "pi5aug18").. then NOW you can also access it with...
192.168.0.180/MyFirstPiWebpage.htm
(That will work from any PC on the same LAN, or even from the Pi itself. Although it will be "the long way 'round the houses" to access it that way, from the Pi: The request will be going out from the Pi to the router, and the reply will also follow that route. But testing that it WILL WORK is worthwhile, even if you wouldn't be doing that, that way, in everyday use, probably.)
So what's my router's WIPA?
Your router is probably doing at least TWO jobs.
1) It is the interface between "the outside world" and at least one computer "inside" your world.
2) It manages your LAN. It allows one computer on the LAN to (within the limits of permissions established) talk to other computers also on your LAN.
When computers on the LAN speak to one another, the process is quite straightforward.
One of the computers talking to "the outside world" is more complex, as we've already discussed. Not only do the messages have to go back and forth, but as they "come in", via the router's WIPA, they have to be routed, via the right LIPA to the right computer on the LAN.
Think of the router as a Janus... the two faced god. One face "talks" to things on the LAN, the other face "talks" to the internet, the WAN.
When the whole business originates from a computer on the LAN, it is relatively straight-forward, as previously discussed: The person sitting at the computer requests a page. The request, stamped with a return address on the LAN goes to the router. The router wraps that inside another "envelope", and forwards it to the right place (as figured out by the DNS "out there in Internetland"), with the router's WIPA on the envelope as return address. The destination webserver figures out what's wanted, and sends it on it's way, still carrying the two levels of return address. The reply reaches the router. The second "return address" comes into play, and the router passes the message from the "outside world" of the WAN to the inside world of the LAN, passes the message to the right computer on the LAN.
Fine.
But how would an "outside" computer get to your webserver, if the outside computer were starting the "conversation"?
It couldn't, unless it (the outside computer) knew, or could obtain via DNS, the WIPA for your router.
This is where DDNS services come in.
Some are free. I pay $5 a month for a service that is a tiny step up from "free". I use the nice people at Dyn.com to get my DDNS service. Sadly, they no longer offer the free package they once did. They have a 7 day trial! If you are serious about this "have a publicly visible webserver", I would start as you mean to go on" and go with dyn.com from the beginning. But you don't need to. Just find someone offering a free DDNS package. Learn a bit. Maybe stay with them, maybe migrate to Dyn.com
Be sure that the service you are signing up with takes care of registering a domain name for you. Most free packages are going to restrict you to a name of their choosing, but that's no big deal. Even my $5 a month doesn't let me choose my name, entirely. I went with "mon277rr.dyndns.org". The first part comes from the fact that my early webserver was deployed to MONitor something. The 277rr was also chosen by me. It says (to me) where the "something" is. The rest (".dyndns.org") was what I am "stuck with" because of where I get my DDNS services.
The "magic" bit
Here's the "magic" which lets DDNS work: Somewhere behind my router's WIPA, a bit of code regularly checks the WIPA that is currently being used to connect my router to the internet. (As it happens, that bit of code is in my router.... but sometimes you have the code in one of the computers on the LAN.)
If, as sometimes happens, my ISP switches my router from one WIPA to another, the code in my system detects that, and sends a message to the DNS (not DDNS) system and says "mon277rr.dyndns.org" is no longer where it was a moment ago. From now on, please, send any messages for that URL to (such and such), the new WIPA for that domain."
Yes, it sounds unreliable and subject to problems over things being behind the time. Yes, in theory, sometimes my webserver is at a WIPA for a while before the DNS system knows that it is no longer at the old WIPA. But it Works. That's all I can tell you.
So... you need to sign up with a DDNS service provider, and you need to get an updater running, in your router, or somewhere inside your LAN.
Nearly there!
When you've done those things, there's just one more step.
If this is the first time you've done this, if you don't have IP Cams, etc, on your system, the following will probably be enough. More on a possibly troublesome "detail" in a moment.
Go into the settings for your router. (This is usually accomplished by sitting down at a computer on the LAN, opening a browser page, and going to 192.168.0.1.
When you've given your router your user id and password (you HAVE changed those from the defaults for your router, haven't you?)... find the "NAT" ("Network Address Translation") or "Port Forwarding" control panel.
Each router's preferences interface is a bit different, but somewhere you will be able to say "When http stuff comes in to the router via the router's WIPA, send it to such and such a computer on the LAN, the one at LIPA such and such." Often, you are asked what protocols you want to allow. "TCP" will usually be "enough". There may be talk of "ports". In a simple case, the "port" is 80, where-ever the preferences pages asks for "port".
Let's say you're DDNS service said that you could have "MyFirstPiAug18.abcDDNSservices.org" as your domain name. And that you've got everything set up correctly.
At this point, you should be able to enter....
http://MyFirstPiAug18.abcDDNSservices.org
...into a browser on ANY machine which is connected to the internet, and get back the "index.html" page from your webpage! (You can probably leave out the "http://" part.)
Enter....
http://MyFirstPiAug18.abcDDNSservices.org/MyPage.htm
...into a browser, and you should get what's on MyPage.htm in the folder that the webserver serves from.
When I said "ANY" machine, I meant just that....
1) It should work on the Pi that is doing the web serving. (The request will go "out" into Internetland, go back to the Pi. Generate a response. Which will itself go out into Internetland and back again to the Pi, and finally end up on your screen!)
2) It should work on a machine on the same LAN as the webserver is on.
3) Should work on a machine connected to the internet, NOT on the LAN the server is on.
The little matter of ports
Ports. Sigh. So far I've hardly mentioned ports.
The default port for HTTP stuff is 80. When someone says...
http://MyFirstPiAug18.abcDDNSservices.org/MyPage.htm
...the system will receive that at the router's WIPA address, and, because that's the way these things work, assume that the traffic is for port 80.
If someone asks for...
http://MyFirstPiAug18.abcDDNSservices.org:1205/MyPage.htm
...they have said: "This is HTTP stuff, but I want you to route it via port 1205.
The "port" thing is sort of a supplemental address. If not specified, 80 is assumed. But you can explicitly specify different ports.
If you wanted the traffic to your webserver to travel in the "outside world" with port 1205 involved, nothing could be easier. When you set up the entries for the NAT, you would say "things coming in for port 1205 are to be passed to such and such a computer on the LAN". You could say, and send it to that LIPA, and port 1205 on that computer. But it might be easier to say "use port 80". If you want the webserver to operate on an unusual port, a port other than 80, you have to set one of the WEBSERVER's preferences.
On the other hand... if you ARE using, say 1205 for things coming in over the internet, it can be hard on the brain to remember...
For requests from outside:MyFirstPiAug18.abcDDNSservices.org:1205 For requests from within the LAN: 192.168.0.[whatever]:80
You could leave the ":80" off, but, if you are like me, you might find yourself adding the :1205 which you have to use when making a request for service from outside the LAN... and it wouldn't work, if you haven't changed the webserver's settings.
That's it for now
I'm sorry... I know there are bits in the above where I could be more clear. I know there are details which could be more clear. I wish I could be there at your side to help you through the long "recipe" leading to the working webserver I hope this will enable you to achieve. But other tasks are demanding my time.
I will close with a reiteration of my earlier statement that you CAN do it, it IS a lot of fun. It DOES open up some great possibilities....
And I wish you every success!
Next!!!
Once you get a webserver working with basic, "plain vanilla" web page serving, you may soon want to get into the extra joys that having php service allows.
Even before that, you can have the "poor man's" "dynamic" web pages....
If your webpage serves up an image, it will come from an image file held in the folder with the .htm file. Change what's in the image file with the name you used in the .htm file, and what appears on the page changes!
Poor man's "dynamic" 2: If you have an app that changes what is in the .htm file, you again get different pages appearing for the same URL.
Go to Webserver3, the next part of this guide. Adding PHP service to your webserver.
Hope that helped?
I hope that was helpful. Getting started is always so tedious. This page was just "a sidebar" off of my main "Getting started with Raspberry Pi" page. Feel free to contact me (see below) with comments, suggestions, questions... save the next reader being confused by something? Please cite this page's URI, if you do: pt0FirstWebserver2.htm.
Footer
Please remember that this material is copyright. (TK Boyd, 2018) There are further notes in this page's parent page.
The search engine is not intelligent. It merely seeks the words you specify. It will not do anything sensible with "What does the 'could not compile' error mean?" It will just return references to pages with "what", "does", "could", "not".... etc.
This page is a "sidebar" to my main discussion of Getting Started with the extraordinary Raspberry Pi.
To email this page's editor, Tom Boyd.... Editor's email address. Suggestions welcome!
Page tested for compliance with INDUSTRY (not MS-only) standards, using the free, publicly accessible validator at validator.w3.org. Mostly passes. There were two "unknown attributes" in Google+ button code. Sigh.
Why does this page cause a script to run? Because of the Google panels, the code for the search button, etc. Why do I mention scripts? Be sure you know all you need to about spyware.
....... P a g e . . . E n d s .....