NetTalk WebServer - Building Secure Web Sites

When dealing with sensitive information, it is recommended that the web site you create be made “Secure”.

TLS or SSL

Secure Sockets Layer (SSL) was a name coined by Netscape who did most of the original work on securing the web. When the protocol was developed further outside of Netscape it was renamed to Transport Layer Security (TLS). The first version of TLS (version 1.0) was an incremental improvement on SSL version 3. (Think of it as a sort of SSL version 3.1)

For many years SSL v3 and TLS 1.0 lived happily side by side. (SSL v2 was deprecated in 2011). However in 2014 SSL v3 was broken, and it was officially deprecated in June 2015. TLS has also been extended resulting in TLS 1.1 and TLS 1.2. TLS 1.3 is currently under development.

Therefore it is more correct to talk about TLS when talking about secure communications than SSL. Of course the SSL name endures (not least as part of the name of various DLL's etc) but as from NetTalk 9 the term TLS is preferred over SSL.

Why Secure?

A Secure site has several benefits:

You can be sure that the site you are talking to is the real version of that site, and not some spoofed version of the site and
The packets that travel between the browser and the server are encrypted. This means that no machine along the way can “sniff” the packets to see the information.
Search Engines (ie Google) are starting to rank secure sites (even information sites) with a higher score than insecure sites.

A secure site uses a Certificate to hold all the relevant information about the site. The Certificate is Signed by a Certificate Authority, which means the certificate is right. Almost all the effort in making a secure site goes into getting the certificates right. Fortunately this only has to be done once, and is simple enough if you follow the directions exactly.

Mixing Secure and Insecure?

Some years ago it was acceptable to have a site that was mostly insecure, but then switched to secure when certain sensitive information was required. Today this is less acceptable and should be avoided if possible.

As from build 9.24, sessions are no longer shared between secure and insecure parts of a site.

While it is still possible to have a mixed-site approach, internally the two are treated as separate sessions - the data from the insecure session is no longer available to the secure site. This is in accordance with best practices and is enforced in modern browsers.

LetsEncrypt

LetsEncrypt is a free certificate provider, where Domain Verified certificates are both automated, and free.

As from NetTalk 10 the LetsEncrypt protocol (known as the ACME protocol) is supported directly in the server.

Changing your Application so that the Server is a Secure Server

There are two ways to make your server secure - at compile time or at runtime. Runtime is the preferred option because it allows you to use insecure settings while developing, but secure options when the server is deployed.

Secure at Runtime

Open the app in the Clarion IDE and go to the WebServer procedure, to the Window Designer
Add a new tab to the sheet control. Call the Tab "Settings"
Populate the NetWebServerSettings control template onto the tab [1]
Compile and Run.
Follow the instructions in Runtime Settings below.

Secure at Compile Time

Go to the WebServer procedure in your app and click on the Extensions button.
Go to the Settings tab of the NetTalk Web Server object.
Set the Host Names [2] for the server as a comma separated list. (You will need a certificate for each host name. [3], [4]) For example;
'capesoft.com,www.capesoft.com'
Set the Listen on Secure Port. The default port for secure servers is 443.
Set the Listen on Insecure Port. The default port is 80. This will redirect all traffic on this port to the secure port.
In the example these files are in the \certificates folder. In the example, the name of this file is Settings.Crt. So in this setting on the template there is 'certificates\settings.crt'. Notice the path is relative to the application folder. If your file is called say www.capesoft.com.crt then you would enter 'certificates\www.capesoft.com.crt'
Set the name of the key file to use. This typically matches the name of the CRT file (used above) except with the extension key.
Click Ok, then Ok, then compile and Run.

Notes

[1] Populating control templates can be tricky when the template contains another sheet control. For best results watch the NetTalk User Group Webinar #185, from 0:12 to 0:15.

[2] From NetTalk 9 Server Name Indication (SNI) support has been added. This means that the Host Name field can contain multiple values as a comma separated list. For more information about SNI see the section below.

[3] Certificates are located in the certificates subfolder under your exe. There's no template setting for this location, but you can set the s_web._SitesQueue.defaults.CertificatesPath field in the WebServer in the Override Default Server Settings embed point.

[4] Certificates require the .crt and .key files. These must have the same name as the domain, plus the crt or key extension. So for the domain www.capesoft.com the files www.capesoft.com.crt and www.capesoft.com.key should be in the certificates folder.

[5] Runtime settings does not mean you have to use LetsEncrypt certificates. Paid-For certificates can be placed in the certificates folder, and used with Runtime settings with no problems.

[6] Runtime settings does not mean the server has to be secure. Using the runtime settings the server can be set up in insecure mode as well.

Runtime Server Settings

Secure Port

This is the port number that the server will listen on for secure traffic. This is usually set to port 443, but can be any port number. If the site is not secure then you can set this field to 0. If this is set to zero then the Insecure port number should be set, but all the other settings on the Security tab can be blank.

Insecure Port

This is the insecure port number. Any incoming traffic on this port will be redirected to the secure port. This is usually port 80 - if the server is secure, and there is another server on this machine already listening on port 80 then you should leave this setting at 0.

Certificates Folder

This is the location of the certificate (.crt and .key) files. By default this is the certificates folder under the exe. You can set the full path to anywhere on the disk where you would like the certificates to be stored. Make sure the program has write-access to the folder if LetsEncrypt support will be used.

Acme Web Folder

This is the location, of the web folder, for whatever server is listening on port 80 on this machine. In order for LetsEncrypt to work there must be a web server listening on port 80 on this machine, either this server (see Insecure Port above) or some other server.
If this server is listening on port 80, then set this to the Web Folder for this server. If it's some other server then set it to the web folder (ie root folder) of the other server.

CA Account

Some unique name to identify this server with LetsEncrypt. Your business name, or generic site name will do fine. For example we use CapeSoft.

Bind to IP Address

Servers can have multiple local IP Addresses. If you want to limit this server to only serve on a single IP address then enter it here. See Binding Apps to an IP address for more.

Domains

A list of domains that are pointing at this server. For example capesoft.com and www.capesoft.com are served by the same server. This list of domains allows one server to serve pages for more than one domain.

Domains with no extension, or the .local extension will also get certificates generated - although not from LetsEncrypt. Self-Signed certificates for these domains will be generated, and placed in the certificates folder.

Testing

If this checkbox is on then certificates will be retrieved from the LetsEncrypt testing server. It is recommended that this option is on until all the certificates have been successfully retrieved. the LetsEncrypt production server is rate-limited, meaning that if you make some mistakes setting it up, you could exceed the rate limit and have to wait a week or so to try again. By setting it up using the testing server you can get all the settings correct, then switch to the production server.

The testing server certificates are untrusted. This means if they are in use, and you access the site with a browser you will get a scary warning. You can proceed for testing. Once you have completed testing, delete the .CRT files in the certificates folder, untick the testing checkbox and click the Certificates button. This will retrieve certificates from the LetsEncrypt production server.

Certificates

Click on this button to see the dates for the existing certificates. If no certificates exist, or the certificates have less than 30 days before expiry, then new certificates will be fetched.

The Testing option above determines if the certificates come from the Test server or the Production server.

Calling your secure server from your browser

When accessing a site from your browser you usually use the form
http://127.0.0.1:88
where 127.0.0.1 is the name of the server, and 88 is the port it is running on.

To access Secure servers you must use the HTTPS protocol. For the example, running on port 881, this is
https://127.0.0.1:881

All Intranet sites (ie sites on your LAN) which are called using a number, or machine name, will generate scary warnings in your browser. This is because the certificate is self-signed. You can add an exception to the browser to allow it to access the site.

Sites hosted on the internet with an internet address, with a correct certificate, will not encounter this sort of problem.

Server Name Indication (SNI)

When a browser connects to a server it desires the server to use a certificate which matches the URL it is connecting to. For example if it is connecting to www.capesoft.com it becomes upset if the server uses a certificate issued to www.clarionshop.com.

Because the connection is made (and hence the certificate chosen) before any data is transferred the server does not know which host name the browser is connecting to, and hence which certificate to use.

This means that it has been impossible for multiple secure sites to share a single IP:Port address. Each server could only use one certificate, and hence different servers were required for different sites.

Server Name Indication (SNI) is used to overcome this problem. It is implemented in all modern browsers (*). It is also implemented in the NetTalk 9 (and later) servers allowing NetTalk Web Servers to use multiple certificates on a single server. SNI basically includes an unencrypted Host name (the "server part" of a URL) when creating the connection, before the connection is secured. Using this name the server is able to decide which certificate to use.

In the server app, in the web server procedure, the Host Names field (Settings / General tab) should be set to either set:domains (if the Runtime Settings Control template is being used) or a comma separated list of domain names. For example;
'www.capesoft.com, api.capesoft.com'

On the client side there is nothing you need to do to implement SNI. The NetTalk WebClient class also supports SNI automatically.

(*) Any version of IE on Windows XP or Windows Server 2003 does not support SNI. Any other browser on XP is fine, and any version of IE7 or later on any other version of Windows is fine. For a full list of supported, and unsupported browsers, see Wikipedia.

Connections made from browsers that do not support SNI will use the first certificate specified on the server. Connections made from browsers using an unrecognized hostname will also default to the first certificate.

Binding Apps to an IP address

Quite apart from any TLS considerations, it’s possible to limit access to the server based on the IP that the server is listening on.

By default the server listens on all the IP addresses that are valid for the server. It will listen on 127.0.0.1 (which means the browser is on the same machine), and it will listen on any network cards, or other network interfaces, installed in the machine.

If you are adding a web interface to a program, and you only want that interface to be accessed from that machine, then you can BIND the server to address 127.0.0.1. None of the other network cards will work.

Another situation where this is handy is if the machine has one network card for the LAN, and another for the Internet. By binding the server to the LAN card you prevent people outside the LAN from accessing the web server. Of course you should be preventing unwanted outside traffic anyway via a Firewall, but this provides an additional level of security.

If you are using the runtime settings control template then the IP address to bind the server to is set at runtime.

Alternatively you can bind a server to a specific IP address at compile time by going to

The WebServer procedure
Extensions
NetTalk Object
Settings tab
Security Tab
Bind Server to only IP: set this to the IP address (of this machine) you wish to bind the program to.

This approach is obviously less flexible than setting it at runtime. In the case of binding to 127.0.0.1 though (so the browser has to be on the same machine as the server) it may be more secure.

If you are making a web interface to a Service (a program that you run on a machine in Service mode) then strongly consider limiting the web interface to 127.0.0.1 . This means only browsers running on this same machine will be able to interact with your service.

Deploying a Secure Web Server

Deploying a secure web server is really no different from deploying a normal web server.

See DeployingAtlsClientOrServer

Using Intermediate Certificates

This section can be ignored if you are using LetsEncrypt support. The Intermediate certificate there is added for you.

Some certificates available for purchase require the use of an intermediate certificate in order for the site to work without error in all browsers. Usually your certificate supplier will note this, and will supply you with an intermediate certificate in a .CRT file.

All you need to do to use the intermediate certificates is to merge them into your .CRT file using a simple text editor. The intermediate certificates MUST come after, your certificate in the file.

Example

-----BEGIN CERTIFICATE-----   <-- this is the site's certificate
MIIE3jCCA8agAwIBAgICAwEwDQYJKoZIhvcNAQEFBQAwYzELMAkGA1UEBhMCVVMx
...
qDTMBqLdElrRhjZkAzVvb3du6/KFUJheqwNTrZEjYx8WnM25sgVjOuH0aBsXBTWV
U+4=
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----   <-- these are the intermediate certificate(s)
WBsUs5iB0QQeyAfJg594RAoYC5jcdnplDQ1tgMQLARzLrUc+cb53S8wGd9D0Vmsf
...
SxOaFIqII6hR8INMqzW/Rn453HWkrugp++85j09VZw==
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
MIIC5zCCAlACAQEwDQYJKoZIhvcNAQEFBQAwgbsxJDAiBgNVBAcTG1ZhbGlDZXJ0
...
IYEZoDJJKPTEjlbVUjP9UNV+mWwD5MlM/Mtsq2azSiGM5bUMMj4QssxsodyamEwC
W/POuZ6lcg5Ktz885hZo+L7tdEy8W9ViH0Pd
-----END CERTIFICATE-----

Importing into the Windows Certificate Store

NetTalk servers use the certificates from the certificates folder. Other servers on the machine may take their certificates directly from the Windows Certificate Store.

The certificates retrieved from LetsEncrypt will work on any server. Indeed a NetTalk server could fetch the certificates, and not actually serve any of the domains.

These two facts taken together are intriguing, and the benefits of automatically placing the certificates in the Windows Store are appealing. Since Cryptonite has the facility to import certificates into the store, if you add Cryptonite to the application then you can add a button to the settings tab which contains the following code;

s_web.acme.SetDomains(set:Domains)
s_web.acme.ImportAllPFX()

When the button is pressed all the certificates for the domains will be repackaged and imported into the Windows store.

Checking if it's Correct

Obviously the easiest way to check if you have got it all working correctly is to open the site in a browser. If you get no warnings when accessing the site in the browser, and the browser has a padlock to indicate the page is secure, then you've got it all right.

There are also a few online services which are able to probe your server to determine if you have set up TLS correctly. However be warned - I've had some mixed results with these. Sometimes they report errors which are clearly incorrect, so evaluate their response very carefully before panicking.

Levels of TLS

TLS (Transport Layer Security) is the general term for the encrypted method for the web. However there are various "versions" of the protocol which all fall under this umbrella. Not surprisingly they're not binary compatible. Typically the browser supports "a lot of them", the server supports "a lot of them" and hopefully there's some overlap.

The various versions of TLS go by different names. SSL version 2 and 3 are now obsolete, and vulnerable, and should no longer be used. TLS 1.0, 1.1 and 1.2 are still in operation.

TLS 1.3 is currently in development and will be supported by NetTalk once it has been finalized. It is not expected to require any changes in your code, once it is available and stable it will be included automatically.

Regardless of which TLS version you are using, there are also a large number of encryption schemes which are supported. Ultimately you can determine which version of TLS your server will use, and which encryption schemes are allowed.

SSLMethod

You can set the .SSLMethod property in the ThisWebServer.Open method in the WebServer procedure.
Possible values are;
NET:SSLMethodSSLv23
NET:SSLMethodSSLv2
NET:SSLMethodSSLv3
NET:SSLMethod3TLS ! SSLv3 and TLS 1.0, 1.1 and 1.2

All the above options have been deprecated in NetTalk 10.

NET:SSLMethodTLS ! Default, TLS 1.0, 1.1 and 1.2
NET:SSLMethodTLS_PCI ! TLS 1.1 or TLS 1.2
NET:SSLMethodTLSv1
NET:SSLMethodTLSv1_1
NET:SSLMethodTLSv1_2

Ciphers

The term "TLS" covers a number of encryption schemes which may be implemented by both the server and the client. If the server and client can agree on a scheme, then the conversation goes ahead.

You can test for the schemes supported by your server using the SSLScan tool.

You can download a Windows version of SSLScan from http://code.google.com/p/sslscan-win/.
A good source for SSLScan documentation is here http://www.titania.co.uk/index.php?option=com_content&view=article&id=56&Itemid=68.

The two tests I recommend running are;

sslscan --no-failed localhost:443

Where localhost and 443 are the server, and port numbers respectively.
This test shows all the Ciphers supported by your server.

For a list of all the ciphers that SSLScan will test, along with the result, use

sslscan localhost:443

The default cipher level will be TLSv1.x High level ciphers only.

It is possible to have complete control over the cipher list you support. the ThisWebServer.SSLCertificateOptions.CiphersAllowed property is set to a Cipher List.

This property is set in the WebServer procedure, in the INIT method, around priority 3000. It should come just after the generated line that sets the ThisWebServer.SSLCertificateOptions.PrivateKeyFile property.

The Cipher List string is a colon-separated list, where + means include, and ! means exclude. The format of the cipher list is documented at http://www.openssl.org/docs/apps/ciphers.html#CIPHER_LIST_FORMAT .

The default Cipher List (currently) looks like this;
ThisWebServer.SSLCertificateOptions.CiphersAllowed = 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS'

Note that this default changes from time to time depending on the current best practice. For the best security leave this setting alone, that way when the default is updated your program updates as well (the next time you compile.)

From build 9.18 NetTalk supports ciphers which provide perfect forward security (ECDH). Not all clients support ECDH, and so those clients can fall back on the DH cipher. The DH cipher requires a file called dh2048.pem, which is in your application folder. You need to deploy this file with your application to get the DH cipher.

If the client does not support ECDH and you do not ship dh2048.pem, then they will fall back to using AES (which is still very good.)

Here's an alternate cipher list;
ThisWebServer.CiphersAllowed = 'DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256'
You may need to add more if you want to support older IE11 and Safari releases.

Troubleshooting and Common Errors

The first thing to do if there is a problem with the secure server is to go to the WebServer procedure, to the NetTalk extension (for the server) and turn off the option Suppress Errors. This will usually quickly highlight what the error is. Once the error has been corrected you should turn Suppress Errors back on.

Some common errors are;

-65: SSL Failed to Load Certificate

Secure sites need a .CRT and a .KEY file. This is usually in the app\certificates folder. If one of these files are missing you'll get this error.

-73: NetTalk Could Not Load TLS DLLs [LibCrypto-1_1.DLL, LibSsl-1_1.DLL, MSVCR120.DLL]

The TLS functionality in NetTalk requires the MSVCR120.DLL, LibCrypto-1_1.DLL and LibSsl-1_1.DLL files. These can be found in your application folder or in the \clarion\accessory\bin folder. Copy these three DLL's to your application folder (and include them in your program install.)
You also need to ship CAROOT.PEM and DH2048.PEM.

Note: Unlike the MSVCR90.DLL, the MSVCR120.DLL should be shipped with your program.

Common Errors when getting a LetsEncrypt certificate

When getting certificates from the LetsEncrypt server the process is logged in the text control on the window. This helps you to see where it might be going wrong. Here are some common messages when the process does not work.

Unable to get certificate - Challenge was invalid

The process of getting a certificate from LE involves your server placing a file on the disk. Then LetsEncrypt will call your server, using the normal web request, to get the file. This is how you prove that you are in control of the actual server at that domain. This is why this process has to run on the actual real server. It can't run on a development machine and succeed.

If you get the error above it means that the LE server failed to get the file.

The log of the process looks something like this (read this from bottom to top);

5. Unable to get certificate - Challenge was invalid
4. Checking Status
3. Notify Server Challenge is Ready
2. LE Server will now fetch http://www.somewhere.com:80/.well-known/acme-challenge/C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
1. Challenge Token Saved C:\somewhere\web\.well-known\acme-challenge\C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0

Debugging this process is straight-forward, because you can check each step. Starting from the bottom;

1. Challenge Token Saved
C:\somewhere\web\.well-known\acme-challenge\C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0

Make sure this file has been created, and it exists. It will contain two hashes, separated be a period. For example;
70NW1-dB3MVucYhV3E0ExrUy3pPD7ca8ZKbHO3nGtVs.Qd6eZFsIb1jZt6orgpYEelEz-rmHppspMi_SCPK8Br0

This file should be in a place where the web server listening on port 80 (either your server, or some other server) can serve it from. If the location is not correct you can adjust it by setting the AcmeWebFolder setting.

2. LE Server will now fetch
http://www.somewhere.com:80/.well-known/acme-challenge/C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0

Go to your browser (preferably on a machine not the server) and enter this URL. If the server is set up correctly then you should see the contents of the file in your browser. If you do not see the file, then you need to revisit step 1, and also check your server to make sure it's listening on port 80. If LE can't fetch this file it will not create the certificate for you.

Hint: If you are using IIS on Port 80, then it may need to be configured to server files with no extension. This is done in the web.config file - for example;

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
<system.webServer>
<staticContent>
<clear />
<mimeMap fileExtension=".*" mimeType="text/json" />
</staticContent>
<handlers>
<clear />
<add name="StaticFile" path="*" verb="*" type="" modules="StaticFileModule,DefaultDocumentModule,DirectoryListingModule" scriptProcessor="" resourceType="Either" requireAccess="Read" allowPathInfo="false" preCondition="" responseBufferLimit="4194304" />
</handlers>
</system.webServer>

</configuration>

CapeSoft NetTalk
Building Secure Web Sites

NetTalk WebServer - Building Secure Web Sites

Introduction

TLS or SSL

Why Secure?

Mixing Secure and Insecure?

LetsEncrypt

Changing your Application so that the Server is a Secure Server

Secure at Runtime

Secure at Compile Time

Notes

Runtime Server Settings

Secure Port

Insecure Port

Certificates Folder

Acme Web Folder

CA Account

Bind to IP Address

Domains

Testing

Certificates

Calling your secure server from your browser

Server Name Indication (SNI)

Binding Apps to an IP address

Deploying a Secure Web Server

Using Intermediate Certificates

Importing into the Windows Certificate Store

Checking if it's Correct

Levels of TLS

SSLMethod

Ciphers

Troubleshooting and Common Errors

-65: SSL Failed to Load Certificate

-73: NetTalk Could Not Load TLS DLLs [LibCrypto-1_1.DLL, LibSsl-1_1.DLL, MSVCR120.DLL]

Common Errors when getting a LetsEncrypt certificate

Unable to get certificate - Challenge was invalid

CapeSoft NetTalkBuilding Secure Web Sites

NetTalk WebServer - Building Secure Web Sites

Introduction

TLS or SSL

Why Secure?

Mixing Secure and Insecure?

LetsEncrypt

Changing your Application so that the Server is a Secure Server

Secure at Runtime

Secure at Compile Time

Notes

Runtime Server Settings

Secure Port

Insecure Port

Certificates Folder

Acme Web Folder

CA Account

Bind to IP Address

Domains

Testing

Certificates

Calling your secure server from your browser

Server Name Indication (SNI)

Binding Apps to an IP address

Deploying a Secure Web Server

Using Intermediate Certificates

Importing into the Windows Certificate Store

Checking if it's Correct

Levels of TLS

SSLMethod

Ciphers

Troubleshooting and Common Errors

-65: SSL Failed to Load Certificate

-73: NetTalk Could Not Load TLS DLLs [LibCrypto-1_1.DLL, LibSsl-1_1.DLL, MSVCR120.DLL]

Common Errors when getting a LetsEncrypt certificate

Unable to get certificate - Challenge was invalid

CapeSoft NetTalk
Building Secure Web Sites