NetTalk WebServer - Building Secure Web Sites
Introduction
When dealing with sensitive information, it is recommended that the web site you create be made “Secure”.
TLS or SSL
Secure Sockets Layer (SSL) was a name coined by Netscape who did most of
the original work on securing the web. When the protocol was developed
further outside of Netscape it was renamed to Transport Layer Security
(TLS). The first version of TLS (version 1.0) was an incremental
improvement on SSL version 3. (Think of it as a sort of SSL version 3.1)
For many years SSL v3 and TLS 1.0 lived happily side by side. (SSL
v2 was deprecated in 2011). However in 2014 SSL v3 was broken, and it
was officially deprecated in June 2015. TLS has also been extended
resulting in TLS 1.1,TLS 1.2. and now TLS 1.3.
TLS 1.0 and TLS
1.1 are considered to be less secure and TLS 1.0 is already banned by
sites conforming to the PCI standard. Both TLS 1.0 and TLS 1.1 will
become obsolete in March 2020 as all browser makes have announced
removal of support for those protocols by that date.
Therefore it is more correct
to talk about TLS when talking about secure communications than SSL. Of
course the SSL name endures (not least as part of the name of various
DLL's etc) but as from NetTalk 9 the term TLS is preferred over SSL.
Why Secure?
A Secure site has several benefits:
- You can be sure that the site you are talking to is the real version of that site, and not some spoofed version of the site and
- The packets that travel between the browser and the server are encrypted. This means that no machine along the way can “sniff” the packets to see the information.
- Search Engines (ie Google) are starting to rank secure sites (even
information sites) with a higher score than insecure sites.
A secure site uses a
Certificate to hold all the relevant information about the site.
The Certificate is
Signed by a
Certificate Authority, which means the certificate is right.
Almost all the effort in making a secure site goes into getting the certificates right.
Fortunately this only has to be done once, and is simple enough if you follow the directions exactly.
Mixing Secure and Insecure?
Some years ago it was acceptable to have a site that was mostly insecure, but then switched to secure when certain sensitive information was required.
Today this is less acceptable and should be avoided if possible.
As from build 9.24, sessions are no longer
shared between secure and insecure parts of a site.While it
is still possible to have a mixed-site approach, internally the two are
treated as separate sessions - the data from the insecure session is no
longer available to the secure site. This is in accordance with best
practices and is
enforced in modern browsers.
LetsEncrypt
LetsEncrypt is a free certificate provider, where Domain Verified certificates are both automated, and free.
As from NetTalk 10 the LetsEncrypt protocol (known as the ACME
protocol) is supported directly in the server. This means that making web
apps secure can done trivially and at no cost. There is no good reason to
create a web app that does not make use of this security.
Changing your Application so that the Server is a Secure Server
There are two ways to make your server secure - at compile time or at runtime. Runtime is the preferred option because it allows you to
use insecure settings while developing, but secure options when the server is deployed.
Secure at Runtime
-
Open the app in the Clarion IDE and go to the WebServer procedure, to the Window Designer
-
Add a new tab to the sheet control. Call the Tab "Settings"
-
Populate the NetWebServerSettings control template onto the tab
[1]
-
Compile and Run.
-
Follow the instructions in Runtime Settings below.
Secure at Compile Time
- Go to the WebServer procedure in your app and click on the Extensions button.
- Go to the Settings tab of the NetTalk Web Server object.
- Set the Host Names [2] for the server
as a comma separated list. (You will need a certificate for each
host name. [3], [4])
For example;
'capesoft.com,www.capesoft.com'
- Set the Listen on Secure Port. The default port for secure servers is 443.
- Set the Listen on Insecure Port. The default port is 80.
This will redirect all traffic on this port to the secure port.
- In the example these files are in the
\certificates folder. In the example, the name of this file is
Settings.Crt. So in this setting on the template there is
'certificates\settings.crt'. Notice the path is relative to the
application folder. If your file is called say
www.capesoft.com.crt then you would enter
'certificates\www.capesoft.com.crt'
- Set the name of the key file to use. This typically matches the name of the CRT file (used above) except with the extension key.
- Click Ok, then Ok, then compile and Run.
Notes
[1]
Populating control templates can be tricky when the template
contains another sheet control. For best results watch the
NetTalk User Group Webinar #185, from 0:12 to 0:15.
[2] From NetTalk 9 Server Name Indication (SNI) support has been added. This means that the
Host Name field can contain multiple values as a
comma separated list. For more information about SNI see the
section below.
[3] Certificates are located in the
certificates subfolder under your exe. There's no template
setting for this location, but you can set the
s_web._SitesQueue.defaults.CertificatesPath field in the
WebServer in the
Override Default Server
Settings embed point. If you are using the Settings Control
template the location can be set at runtime (to anywhere).
[4]
Certificates require the
.crt and
.key
files. These must have the same name as the domain, plus the
crt or
key
extension. So for the domain www.capesoft.com the files
www.capesoft.com.crt and
www.capesoft.com.key should be in the
certificates folder.
[5] Runtime settings does not mean you
have to use LetsEncrypt certificates. Paid-For certificates can be
placed in the certificates folder, and used with Runtime settings
with no problems.
[6] Runtime settings does not mean the
server
has to be secure at all. Using the runtime settings the server can
be set up in insecure mode as well.
Runtime Server Settings
Secure Port
This is the port number that the server will listen on for
secure traffic. This is usually set to port 443, but can be any
port number. If the site is not secure then you can set this
field to 0. If this is set to zero then the Insecure port number
should be set, but all the other settings on the Security tab
can be blank.
Insecure Port
This is the insecure port number. Any incoming traffic on this
port will be redirected to the secure port. This is usually port
80 - if the server is secure, and there is another server on
this machine already listening on port 80 then you should leave
this setting at 0.
Certificates Folder
This is the location of the certificate (.crt and .key) files.
By default this is the certificates folder under the exe. You
can set the full path to anywhere on the disk where you would
like the certificates to be stored. Make sure the program has
write-access to the folder if LetsEncrypt support will be used.
Acme Web Folder
This is the location, of the web folder, for whatever server is
listening on port 80 on this machine. In order for LetsEncrypt
to work there must be a web server listening on port 80 on this
machine, either this server (see
Insecure Port above) or some other server.
If this server
is listening on port 80, then set this to the Web Folder for
this server. If it's some other server then set it to the web
folder (ie root folder) of the
other server.
CA Account
Some unique name to identify this server with LetsEncrypt. Your
business name, or generic site name will do fine. For example we
use CapeSoft.
Bind to IP Address
Servers can have multiple local IP Addresses. If you want to
limit this server to only serve on a single IP address then
enter it here. See
Binding Apps to an IP address
for more.
Domains
A list of domains that are pointing at this server. For example
capesoft.com and
www.capesoft.com are served by the same server. This list of
domains allows one server to serve pages for more than one
domain.
Domains with no extension, or the
.local extension will also get
certificates generated - although not from LetsEncrypt.
Self-Signed certificates for these domains will be generated,
and placed in the certificates folder.
Testing
If this checkbox is on then certificates will be retrieved from
the LetsEncrypt testing server. It is recommended that this
option is on until all the certificates have been successfully
retrieved. the LetsEncrypt production server is rate-limited,
meaning that if you make some mistakes setting it up, you could
exceed the rate limit and have to wait a week or so to try
again. By setting it up using the testing server you can get all
the settings correct, then switch to the production server.
The testing server certificates are untrusted. This means if
they are in use, and you access the site with a browser you will
get a scary warning. You can proceed for testing. Once you have
completed testing, delete the .CRT files in the certificates
folder, untick the testing checkbox and click the Certificates
button. This will retrieve certificates from the LetsEncrypt
production server.
Certificates
Click on this button to see the dates for the existing
certificates. If no certificates exist, or the certificates have
less than 30 days before expiry, then new certificates will be
fetched.
The
Testing
option above determines if the certificates come from the Test
server or the Production server.
Calling your secure server from your browser
When accessing a site from your browser you usually use the form
http://127.0.0.1:88
where 127.0.0.1 is the name of the server, and 88 is the port it is running on.
To access Secure servers you must use the HTTPS protocol. For the example, running on port 881, this is
https://127.0.0.1:881
All Intranet sites (ie sites on your LAN) which are called using a
number, or machine name, will generate scary warnings in your browser. This
is because the certificate is self-signed. You can add an exception to the
browser to allow it to access the site.
Sites hosted on the internet with an internet address, with a correct
certificate, will not encounter this sort of problem.
Server Name Indication (SNI)
When a browser connects to a server it desires the server to use a
certificate which matches the URL it is connecting to. For example if it is
connecting to
www.capesoft.com it
becomes upset if the server uses a certificate issued to
www.clarionshop.com.
Because
the connection is made (and hence the certificate chosen) before any data is
transferred the server does not know which host name the browser is
connecting to, and hence which certificate to use.
This means that it
has been impossible for multiple secure sites to share a single IP:Port
address. Each server could only use one certificate, and hence different
servers were required for different sites.
Server Name Indication
(SNI) is used to overcome this problem. It is implemented in all modern
browsers (*). It is also implemented in the NetTalk 9 (and later) servers allowing
NetTalk Web Servers to use multiple certificates on a single server. SNI
basically includes an unencrypted Host name (the "server part" of a URL)
when creating the connection, before the connection is secured. Using this
name the server is able to decide which certificate to use.
In the
server app, in the web server procedure, the Host Names field (Settings /
General tab) should be set to either
set:domains
(if the Runtime Settings Control template is being used) or a comma
separated list of domain names. For example;
'www.capesoft.com,
api.capesoft.com'
On the client side there is nothing you need to do to implement SNI. The
NetTalk WebClient class also supports SNI automatically.
(*) Any
version of IE on Windows XP or Windows Server 2003 does not support SNI. Any
other browser on XP is fine, and any version of IE7 or later on any other
version of Windows is fine. For a full list of supported, and unsupported
browsers, see
Wikipedia.
Connections made from browsers that do not support SNI
will use the first certificate specified on the server. Connections made
from browsers using an unrecognized hostname will also default to the first
certificate.
Binding Apps to an IP address
Quite apart from any TLS considerations, it’s possible to limit access to the server based on the IP that the server is listening on.
By default the server listens on all the IP addresses that are valid for the server. It will listen on 127.0.0.1 (which means the browser is on the same machine), and it will listen on any network cards, or other network interfaces, installed in the machine.
If you are adding a web interface to a program, and you only want that interface to be accessed from that machine, then you can BIND the server to address 127.0.0.1. None of the other network cards will work.
Another situation where this is handy is if the machine has one network card for the LAN, and another for the Internet. By binding the server to the LAN card you prevent people outside the LAN from accessing the web server.
Of course you should be preventing unwanted outside traffic anyway via a Firewall, but this provides an additional level of security.
If you are using the runtime settings control template then the IP
address to bind the server to is set at runtime.
Alternatively you can bind a server to a specific IP address at compile
time by going to
- The WebServer procedure
- Extensions
- NetTalk Object
- Settings tab
- Security Tab
- Bind Server to only IP: set this to the IP address (of this
machine) you wish to bind the program to.
This approach is obviously less flexible than setting it at runtime. In the
case of binding to 127.0.0.1 though (so the browser has to be on the same
machine as the server) it may be more secure.
If you are making a web interface to a Service (a program that you run on a machine in Service mode) then strongly consider limiting the web interface to 127.0.0.1 . This means only browsers running on this same machine will be able to interact with your service.
Deploying a Secure Web Server
Using Intermediate Certificates
This section can be ignored if you are using
LetsEncrypt support. The Intermediate certificate there is added for you.
Some certificates available for purchase
require the use of an intermediate certificate in order for the site to work
without error in all browsers. Usually your certificate supplier will note this,
and will supply you with an intermediate certificate in a .CRT file.
All you
need to do to use the intermediate certificates is to merge them into your
.CRT
file using a simple text editor. The intermediate certificates
MUST come
after, your certificate in the file.
Example
-----BEGIN
CERTIFICATE-----
MIIE3jCCA8agAwIBAgICAwEwDQYJKoZIhvcNAQEFBQAwYzELMAkGA1UEBhMCVVMx
...
qDTMBqLdElrRhjZkAzVvb3du6/KFUJheqwNTrZEjYx8WnM25sgVjOuH0aBsXBTWV
U+4=
-----END CERTIFICATE-----
-----BEGIN
CERTIFICATE-----
WBsUs5iB0QQeyAfJg594RAoYC5jcdnplDQ1tgMQLARzLrUc+cb53S8wGd9D0Vmsf
...
SxOaFIqII6hR8INMqzW/Rn453HWkrugp++85j09VZw==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIC5zCCAlACAQEwDQYJKoZIhvcNAQEFBQAwgbsxJDAiBgNVBAcTG1ZhbGlDZXJ0
...
IYEZoDJJKPTEjlbVUjP9UNV+mWwD5MlM/Mtsq2azSiGM5bUMMj4QssxsodyamEwC
W/POuZ6lcg5Ktz885hZo+L7tdEy8W9ViH0Pd
-----END CERTIFICATE-----
Importing into the Windows Certificate Store
NetTalk servers use the certificates from the certificates folder. Other
servers on the machine may take their certificates directly from the Windows
Certificate Store.
The certificates retrieved from LetsEncrypt will
work on any server. Indeed a NetTalk server could fetch the certificates,
and not actually serve any of the domains.
These two facts taken
together are intriguing, and the benefits of automatically placing the
certificates in the Windows Store are appealing. Since Cryptonite has the
facility to import certificates into the store, if you add Cryptonite to the
application then you can add a button to the settings tab which contains the
following code;
s_web.acme.SetDomains(set:Domains)
s_web.acme.ImportAllPFX()
When the button is pressed all the
certificates for the domains will be repackaged and imported into the
Windows store.
Checking if it's Correct
Obviously the easiest way to check if you have got it all working correctly is to open the site
in a browser. If you get no warnings when accessing the site in the browser, and the browser has a padlock
to indicate the page is secure, then you've got it all right.
There are also a few online services which are able to probe your server to determine if you have set up
TLS correctly. However be warned - I've had some mixed results with these. Sometimes they report errors
which are clearly incorrect, so evaluate their response very carefully before
panicking.
Levels of TLS
TLS (Transport Layer Security) is the general term for the encrypted method for
the web. However there are various "versions" of the protocol which all fall
under this umbrella. Not surprisingly they're not binary compatible.
Typically the browser supports "a lot of them", the server supports "a lot
of them" and hopefully there's some overlap.
The various versions of
TLS go by different names. SSL version 2 and 3 are now obsolete, and vulnerable, and
should no longer be used. TLS 1.0, 1.1, 1.2 and 1.3 are still in operation.
TLS 1.0 and TLS 1.1 are scheduled to be deprecated in March 2020.
Regardless of which
TLS version you
are using, there are also a large number of encryption schemes which are
supported. Ultimately you can determine which version of TLS your server
will use, and which encryption schemes are allowed.
SSLMethod
You can set the .SSLMethod property in the
ThisWebServer.Open method in the
WebServer procedure.
Possible values are;
NET:SSLMethodTLS
NET:SSLMethodTLS_PCITLS 1.1 or higherNET:SSLMethodTLSv1
NET:SSLMethodTLSv1_1
NET:SSLMethodTLSv1_2
NET:SSLMethodTLSv1_3
NET:SSLMethodTLS_MIN1_1
NET:SSLMethodTLS_MIN1_2
NET:SSLMethodTLS_MIN1_3
Ciphers
The term "TLS" covers a number of encryption schemes which may be
implemented by both the server and the client. If the server and client
can agree on a scheme, then the conversation goes ahead.
You can
test for the schemes supported by your server using the SSLScan tool.
You can download a Windows version of SSLScan from
http://code.google.com/p/sslscan-win/.
Some documentation for the SSLScan tool can be found at
https://www.mankier.com/1/sslscan
The two tests I recommend running are;
sslscan --no-failed localhost:443Where localhost and 443
are the server, and port numbers respectively.
This test shows all
the Ciphers supported by your server.
For a list of all the
ciphers that SSLScan will test, along with the result, use
sslscan localhost:443The default
cipher level will be TLSv1.x High level ciphers only.
It is possible to have complete
control over the cipher list you support. the
ThisWebServer.SSLCertificateOptions.CiphersAllowed property is
set to a
Cipher List.
This property is set in the
WebServer procedure, in the
INIT method, around priority 3000. It should
come just after the generated line that sets the
ThisWebServer.SSLCertificateOptions.PrivateKeyFile property.
The Cipher List string is a colon-separated list, where + means
include, and ! means exclude. The format of the cipher list is
documented at
http://www.openssl.org/docs/apps/ciphers.html#CIPHER_LIST_FORMAT .
The default Cipher List (currently) looks like this;
ThisWebServer.SSLCertificateOptions.CiphersAllowed =
'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS!RSA'
Note that this default changes from time to time depending on the
current best practice. For the best security leave this setting alone,
that way when the default is updated your program updates as well (the
next time you compile.)
From build 9.18 NetTalk supports ciphers which provide perfect
forward security (ECDH). Not all clients support ECDH, and so those
clients can fall back on the DH cipher. The DH cipher requires a file
called dh2048.pem, which is in your application folder. You need to
deploy this file with your application to get the DH cipher.
If
the client does not support ECDH and you do not ship dh2048.pem, then
they will fall back to using AES (which is still very good.)
Here's an example alternate cipher list;
ThisWebServer.CiphersAllowed =
'DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256'
You may need to add more if you want to support older IE11 and Safari
releases.
Troubleshooting and Common Errors
The first thing to do if there is a problem with the secure server is to go
to the WebServer procedure, to the NetTalk extension (for the server) and
turn off the option
Suppress Errors. This will usually
quickly highlight what the error is. Once the error has been corrected you
should turn Suppress Errors back on.
Some common errors are;
-65: SSL Failed to Load Certificate
Secure sites need a .CRT and a
.KEY file. This is usually in the
app\certificates folder. If one of these files are missing you'll get
this error.
-73: NetTalk Could Not Load TLS DLLs [LibCrypto-1_1.DLL, LibSsl-1_1.DLL]
The TLS functionality in NetTalk requires the
LibCrypto-1_1.DLL and
LibSsl-1_1.DLL files. These can
be found in your application folder or in the
\clarion\accessory\bin folder.
Copy these two DLL's to your application folder (and include them in
your program install.)
You also need to ship
CAROOT.PEM and
DH2048.PEM.
You may also need to install the
Visual Studio 2017 (x86) runtime
on the target machine. On most desktop machines this will be installed
already, but on Server machines you will most-likely need to install it.
Common Errors when getting a LetsEncrypt certificate
When getting certificates from the LetsEncrypt server the process is logged in the text control on the window. This helps
you to see where it might be going wrong. Here are some common messages when the process does not work.
VCRuntime140.DLL Missing
Unable to get certificate - Challenge was invalid
The process of getting a certificate from LE involves your server placing a file on the disk. Then LetsEncrypt will call your
server, using the normal web request, to get the file. This is how you prove that you are in control of the actual server at that domain.
This is why this process has to run on the actual real server. It
can't run on a development machine and succeed.
If you get the error above it means that the LE server failed to get the file.
The log of the process looks something like this (read this
from bottom to top);
5. Unable to get certificate - Challenge
was invalid
4. Checking Status
3. Notify Server Challenge is
Ready
2. LE Server will now fetch
http://www.somewhere.com:80/.well-known/acme-challenge/C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
1. Challenge Token Saved
C:\somewhere\web\.well-known\acme-challenge\C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
Debugging this process is straight-forward, because you can
check each step. Starting from the bottom;
1. Challenge Token Saved
C:\somewhere\web\.well-known\acme-challenge\C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
Make sure this file has been created, and it exists. It
will contain two hashes, separated be a period. For example;
70NW1-dB3MVucYhV3E0ExrUy3pPD7ca8ZKbHO3nGtVs.Qd6eZFsIb1jZt6orgpYEelEz-rmHppspMi_SCPK8Br0
This file should be in a place where the web server listening on
port 80 (either your server, or some other server) can serve it
from. If the location is not correct you can adjust it by setting
the
AcmeWebFolder setting.
2. LE Server will now fetch
http://www.somewhere.com:80/.well-known/acme-challenge/C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
Go to your browser (preferably on a machine not the server) and
enter this URL. If the server is set up correctly then you should
see the contents of the file in your browser. If you do not see the
file, then you need to revisit step 1, and also check your server to
make sure it's listening on port 80. If LE can't fetch this file it
will not create the certificate for you.
Hint: If you are using IIS on Port
80, then it may need to be configured to server files with no
extension. This is done in the
web.config file - for example;
<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<staticContent>
<clear />
<mimeMap fileExtension=".*"
mimeType="text/json" />
</staticContent>
<handlers>
<clear />
<add name="StaticFile"
path="*" verb="*" type="" modules="StaticFileModule,DefaultDocumentModule,DirectoryListingModule"
scriptProcessor="" resourceType="Either" requireAccess="Read"
allowPathInfo="false" preCondition="" responseBufferLimit="4194304"
/>
</handlers>
</system.webServer>
</configuration>
[End of this document]