NetTalk WebServer - Building Secure Web Sites
    
    
    Introduction
     When dealing with sensitive information, it is
      recommended that the web site you create be made “Secure”. 
      TLS or SSL
       Secure Sockets Layer (SSL) was a name coined by
        Netscape who did most of the original work on securing the web. When the
        protocol was developed further outside of Netscape it was renamed to
        Transport Layer Security (TLS). The first version of TLS (version 1.0)
        was an incremental improvement on SSL version 3. (Think of it as a
        sort-of SSL version 3.1)
        
        For many years SSL v3 and TLS 1.0 lived happily side by side. (SSL v2
        was deprecated in 2011). However in 2014 SSL v3 was broken, and it was
        officially deprecated in June 2015. TLS has also been extended resulting
        in TLS 1.1,TLS 1.2. and now TLS 1.3.
        
        TLS 1.0 and TLS 1.1 are considered to be less secure and TLS 1.0 is
        already banned by sites conforming to the PCI standard. Both TLS 1.0 and
        TLS 1.1 became obsolete in March 2020 as all browser makers removed
        support for those protocols around that time.
        
        Therefore it is more correct to talk about TLS when talking about secure
        communications than SSL. Of course the SSL name endures (not least as
        part of the name of various DLL's etc) but as from NetTalk 9 the term
        TLS is preferred over SSL.
      
      Why Secure?
       A Secure site has several benefits: 
        
          - You can be sure that the site you are talking to is the real
            version of that site, and not some spoofed version of the site. 
- The packets that travel between the browser and the server are
            encrypted. This means that no machine along the way can “sniff” the
            packets to see the information. 
- Search Engines (eg Google, Bing etc) are starting to rank secure
            sites (even information sites) with a higher score than insecure
            sites.
        A secure site uses a 
Certificate to hold all
        the relevant information about the site. The Certificate is 
Signed
        by a 
Certificate Authority,
        which means the certificate is right. Almost all the effort in making a
        secure site goes into getting the certificates right. Fortunately this
        only has to be done once, and is simple enough if you follow the
        directions exactly.  
      
Mixing Secure and Insecure?
       Some years ago it was acceptable to have a site that
        was mostly insecure, but then switched to secure when certain sensitive
        information was required. Today this is less acceptable and should be
        avoided if possible.
        
        As from NetTalk build 9.24, sessions are no
          longer shared between secure and insecure parts of a site.
        
        While it is still possible to have a mixed-site approach, internally the
        two are treated as separate sessions - the data from the insecure
        session is no longer available to the secure site. This is in accordance
        with best practices and is 
 enforced in modern browsers.
      
Certificates
       TLS works by having the server make use of a
        Certificate. If this certificate is issued by a Certificate Authority
        (CA) then it is considered to be "trusted". If it is simply generated by
        the server (known as Self-Signed) then it is
        considered to be untrusted (and hence less-secure), and a scary warning
        will appear in the browser. In some cases (but not all) the user is able
        to click through to the site anyway. Outside of some very narrow use
        cases Self-Signed certificates are not a good solution.
        
      
    LetsEncrypt
     LetsEncrypt is a free certificate provider, where
      Domain Verified (DV) certificates are both automated, and free. The web
      has embraced this model, 
      
      As from NetTalk 10 the LetsEncrypt protocol (known as the ACME protocol)
      is supported directly in the server. This means that making web apps
      secure can done trivially and at no cost. There is no good reason to
      create a web app that does not make use of this security. Most sites today
      make use of HTTP security.
      
      Over the years there has been much talk about Let'sEncrypt certificates as
      being somehow different to, or less secure, than certificates provided by
      other certificate authorities. This is categorically not the case. All
      certificates are the same 
[1] - and are simply
      signed by a Certificate Authority. Which certificate authority you choose
      makes no difference to the certificate itself. 
[2]
      
      [1] Certificates come in various flavors. The
      simplest is Doman Validation (DV) certificates. These ask the server to
      prove it has control of the domain. If it can show that it has control,
      then a certificate is issued. Organization Validation (OV) and Extended
      Validation (EV) certificates require a lot more validation by the
      Certificate Authority before they issue the certificate. This takes a lot
      of time, and costs money. It should be noted that OV and EV certificates
      offer 
no tangible security benefits to end users, and
      their use is largely 
falling away. They exist primarily as a way for CAs
      to earn money from naive customers.
      
      
[2] NetTalk supports using certificates, of any kind
      (DV, OV or EV) from any CA. If you like spending money to get
      certificates, then feel free to continue to do so.
      
      
 Challenges 
       In order to prove control of the domain the server
        can perform one of several tasks. These tasks are known as Challenges.
        Up to NetTalk 14 the only challenge supported was 
HTTP.
        This means that the server (or some other program on the server
        computer) MUST be listening on Port 80. It also limits certificates to
        those servers that are directly visible to the internet.
        
        NetTalk 14 introduces support for another challenge method - 
DNS.
        This method is restricted to cases where your DNS registrar supports an
        API, but in cases where it does, it allows for providing certificates
        for LAN servers, and servers where opening port 80 
is
          not an option.  You'll find more information on this new
        method at 
Adding Support for DNS
          Challenge to a WebServer.
        
        NetTalk also adds another option - the option to use an intermediate DNS
        API server. This adds a layer of protection to you the developer (by not
        exposing your DNS API credentials to the customer machine.) It is not
        required to use this, it's a nice-to-have. This is discussed at 
Getting
          a Certificate via an API.
Changing your Application so that the
      Server is a Secure Server
     There are two ways to make your server secure - at 
compile
        time or at 
run time. Run time is the preferred option
      because it allows you to use insecure settings while developing, but
      secure options when the server is deployed.
      
Secure at Runtime
      
        
          -  Open the app in the Clarion IDE and go to the WebServer
            procedure, to the Window Designer 
-  Add a new tab to the sheet control. Call the Tab "Settings"
-  Populate the NetWebServerSettings control template onto the tab [1]
-  Compile and Run. 
-  Follow the instructions in Runtime
              Server Settings below.
 
      Secure at Compile Time
      
        
          - Go to the WebServer procedure in your app and click on the
            Extensions button. 
- Go to the Settings tab of the NetTalk Web Server  object.
- Set the Host Names [2] for
            the server as a comma separated list. (You will need a certificate
            for each host name. [3], [4]
            ,[7])
 For example;
 'capesoft.com,capesoft.com'
 
- Set the Listen on Secure Port. The default port
            for secure servers is 443. 
- Set the Listen on Insecure Port. The default
            port is 80. (It MUST be 80 to use LetsEncrypt HTTP Challenges). This
            will redirect all traffic on this port to the secure port. (Worried
            about port 80? see Port 80 Concerns.)
-  Click Ok, then Ok, then compile and
            Run.  
 
      Notes
       [1] Populating control
        templates can be tricky when the template contains another sheet
        control. For best results watch the 
 NetTalk User Group Webinar #185, from 0:12 to
        0:15.
         
        
[2] From NetTalk 9 Server Name Indication (SNI)
        support has been added. This means that the Host Name field can contain
        multiple values as a comma separated list. For more information about
        SNI see the 
section below.
        
        
[3] Certificates are located in the 
certificates
          subfolder under your exe. There's no template setting for this
        location, but you can set the 
          s_web._SitesQueue.defaults.CertificatesPath field in the
        WebServer in the 
Override Default Server Settings
        embed point. If you are using the Settings Control template the location
        can be set at runtime (to anywhere).
        
        
[4] Certificates require the 
.crt
        and 
.key files. These must have the same
        name as the domain, plus the 
crt or 
key extension. So for the domain 
capesoft.com
        the files 
capesoft.com.crt and 
capesoft.com.key should be in the
        certificates folder.
        
        [5] Runtime settings does not mean you
 have to use LetsEncrypt
        certificates. Paid-For certificates can be placed in the certificates
        folder, and used with Runtime settings with no problems. 
        
        [6] Runtime settings does not mean the server 
has to be secure
        at all. Using the runtime settings the server can be set up in insecure
        mode as well.
        
        
[7] Certificates can be supplied in many forms. If
        you get your certificates via some other method, then you need to get it
        in the correct form, or convert it to the correct form (Google will help
        with that.) The correct form is CRT and KEY files. The Key can contain a
        password, or not, as you desire.
        
        
      
 
    Runtime Server Settings
    
    Calling your secure server from your
      browser
     When accessing a site from your browser you usually
      use the form
      
      http://127.0.0.1:88 
      
      where 127.0.0.1 is the name of the server, and 88 is the port it is
      running on. The default for HTTP is port 80. If you listen on port 80 you
      don't need the :port number part of the address.
      
      To access Secure servers you must use the HTTPS protocol. For the example,
      running on port 881, this is 
      
        https://127.0.0.1:881 
      
      The default for HTTPS is port 443. If you listen on port 443 you don't
      need the :port number part of the address.
      
      All Intranet sites (ie sites on your LAN) which are called using an IP
      address, or machine name, will generate scary warnings in your browser.
      This is because the certificate is self-signed. You can add an exception
      to the browser to allow it to access the site. Sites hosted on the
      internet with an internet address, with a correct certificate, will not
      encounter this sort of problem. 
      
      To host on the intranet, and not get a security warning, see 
LetsEncrypt
        DNS Challenge.
Server Name Indication (SNI)
     When a browser connects to a server it desires the
      server to use a certificate which matches the URL it is connecting to. For
      example if it is connecting to 
capesoft.com
      it becomes upset if the server uses a certificate issued to 
www.clarionshop.com.
      
      Because the connection is made, and hence the certificate chosen, before
      any data is transferred, the server does not know which host name the
      browser is connecting to, and hence which certificate to use.
      
      This means that it 
was impossible for multiple secure sites to
      share a single IP:Port address. Each server could only use one
      certificate, and hence different servers were required for different
      sites.
      
      Server Name Indication (SNI) is used to overcome this problem. It is
      implemented in all modern browsers 
[1]. It is
      also implemented in the NetTalk 9 (and later) servers allowing NetTalk Web
      Servers to use multiple certificates on a single server. SNI basically
      includes an unencrypted Host name (the "server part" of a URL) when
      creating the connection, before the connection is secured. Using this name
      the server is able to decide which certificate to use. 
      
      In the server app, in the web server procedure, the Host Names field
      (Settings / General tab) should be set to either 
set:domains
      (if the 
Runtime Settings Control template
      is being used) or a comma separated list of domain names. For example;
      
'capesoft.com, api.capesoft.com' 
      
      On the client side there is nothing you need to do to implement SNI. The
      NetTalk WebClient class also supports SNI automatically. 
      
      
[1] Any version of IE on Windows XP or Windows
      Server 2003 does not support SNI. Any other browser on XP is fine, and any
      version of IE7 or later on any other version of Windows is fine. For a
      full list of supported, and unsupported browsers, see 
 Wikipedia.
      
      Connections made from browsers that do not support SNI will use the first
      certificate specified on the server. Connections made from browsers using
      an unrecognized hostname will also default to the first certificate.
Binding Servers to an IP address
     Quite apart from any TLS considerations, it’s possible
      to limit access to the server based on the IP that the server is listening
      on.
      
By default the server listens on all the IPv4 or IPv6
        addresses (not both) that are valid for the server. For IPv4 it will
        listen on 127.0.0.n (which means
        the browser is on the same machine), and it will listen on any network
        cards, or other network interfaces, installed in the machine. For IPv6
        it listens on ::1.
      If you are adding a web interface to a program, and you only want that
        interface to be accessed from that machine, then you can BIND the server
        to address 127.0.0.1 or ::1. 
        None of the other network cards will work. 
      Another situation where this is handy is if the machine has one network
        card for the LAN, and another for the Internet. By binding the server to
        the LAN card you prevent people outside the LAN from accessing the web
        server. Of course you should be preventing unwanted outside traffic
        anyway via a Firewall, but this provides an additional level of
        security. 
      If you are using the Runtime
          Settings control template then the IP address to bind the server
        to is set at run time.(Whether you are using IPv4 or IPv6 is also set at
        run time)
      Alternatively you can bind a server to a specific IP address at compile
        time by going to 
      
        - The WebServer procedure
- Extensions
- NetTalk Object
- Settings tab
- Security Tab 
- Bind Server to IP Address.  Set this to the IP
          address (of this machine) you wish to bind the program to.
This approach is obviously less flexible than setting it at run time.
        In the case of binding to 127.0.0.1 though (so the browser has to be on
        the same machine as the server) it may be more secure.
      If you are making a web interface to a Service program (a program that
        you run on a machine in Service mode) then strongly consider limiting
        the web interface to 127.0.0.1 . This means only browsers running on
        this same machine will be able to interact with your service. 
    Deploying
      a Secure Web Server
    
    Using Other Certificates
     If you, or your customer, wishes to use a certificate
      manually provided to you from some other Certificate Authority, then it
      will work as easily as an LE certificate. 
      
      Certificates can be provided (by your provider) in any number of formats
      (PEM, PFX, CRT/KEY being the most common.) NetTalk makes use of the
      certificate in CRT and KEY files. You can use the OpenSSL.Exe program to
      convert from one format to another. (Google for instructions on
      conversions.) 
      
      The CRT and KEY files should be placed in the Certificates folder, and the
      server restarted.
      
      Remember that the certificates will need to be periodically replaced from
      time to time as they expire.
      
      If the KEY file has a password, then make use of the 
Passwords
      field to set the password to the certificate.
Using Intermediate
      Certificates
     This section can be
        ignored if you are using LetsEncrypt support. The Intermediate
        certificate for LetsEncrypt is added for you.
      
      Some certificates available for purchase require the use of an
      intermediate certificate in order for the site to work without error in
      all browsers. Usually your certificate supplier will note this, and will
      supply you with an intermediate certificate in a .CRT file.
      
All you need to do to use the intermediate certificates is to merge
        them into your .CRT file using a
        simple text editor. The intermediate certificates MUST
        come after, your certificate in the file.
      Example
      -----BEGIN CERTIFICATE-----   
          MIIE3jCCA8agAwIBAgICAwEwDQYJKoZIhvcNAQEFBQAwYzELMAkGA1UEBhMCVVMx
          ...
          qDTMBqLdElrRhjZkAzVvb3du6/KFUJheqwNTrZEjYx8WnM25sgVjOuH0aBsXBTWV
          U+4=
          -----END CERTIFICATE-----     
          -----BEGIN CERTIFICATE-----   
          WBsUs5iB0QQeyAfJg594RAoYC5jcdnplDQ1tgMQLARzLrUc+cb53S8wGd9D0Vmsf
          ...
          SxOaFIqII6hR8INMqzW/Rn453HWkrugp++85j09VZw==
          -----END CERTIFICATE-----
          -----BEGIN CERTIFICATE-----
          MIIC5zCCAlACAQEwDQYJKoZIhvcNAQEFBQAwgbsxJDAiBgNVBAcTG1ZhbGlDZXJ0
          ...
          IYEZoDJJKPTEjlbVUjP9UNV+mWwD5MlM/Mtsq2azSiGM5bUMMj4QssxsodyamEwC
          W/POuZ6lcg5Ktz885hZo+L7tdEy8W9ViH0Pd
          -----END CERTIFICATE-----
         
     
    Importing into the Windows Certificate Store
     NetTalk servers use the certificates from the
      certificates folder. Other servers on the machine may take their
      certificates directly from the Windows Certificate Store (but NetTalk does
      not).
      
      It's worth noting that the certificates retrieved from LetsEncrypt will
      work on any server. Indeed a NetTalk program could just fetch the
      certificates, and not actually serve any of the domains. 
      
      These two facts taken together are intriguing, and the benefits of
      automatically placing the certificates in the Windows Store are appealing.
      Since Cryptonite has the facility to import certificates into the store,
      if you add Cryptonite to the application then you can add a button to the
      settings tab which contains the following code;
      
      s_web.acme.SetDomains(set:Domains)
      s_web.acme.SetPasswords(set:Passwords)
        s_web.acme.ImportAllPFX()
      
      When the button is pressed all the certificates for the domains will be
      repackaged and imported into the Windows store.
    
    Checking if it's Correct
     Obviously the easiest way to check if you have got it
      all working correctly is to open the site in a browser. If you get no
      warnings when accessing the site in the browser, and the browser has a
      padlock to indicate the page is secure, then you've got it all right.
      
      There are also a few online services which are able to probe your server
      to determine if you have set up TLS correctly. However be warned - I've
      had some mixed results with these. Sometimes they report errors which are
      clearly incorrect, so evaluate their response very carefully before
      panicking.
      
    Levels of TLS
     TLS (Transport Layer Security) is the general term for
      the encrypted methodology for the web. However there are various
      "versions" of the protocol which all fall under this umbrella. Not
      surprisingly they're not binary compatible. Typically the browser supports
      "a lot of them", the server supports "a lot of them" and hopefully there's
      some overlap.
      
      The various versions of TLS go by different names. SSL version 2  and
      3 are now obsolete, and vulnerable, and should no longer be used. TLS 1.0,
      1.1 have also been more recently deprecated.  TLS 1.2 and 1.3 are
      still in operation.
      
      Regardless of which TLS version you are using, there are also a large
      number of encryption schemes which are supported. Ultimately you can
      determine which version of TLS your server will use, and which encryption
      schemes are allowed.
      
TLSMethod (formerly SSLMethod)
       You can set the .TLSMethod
        property in the ThisWebServer.Open method
        in the WebServer procedure. 
        Possible values are;
         NET:SSLMethodTLS     
           
        NET:SSLMethodTLS_PCITLS 1.1 or higherNET:SSLMethodTLSv1      
           
          NET:SSLMethodTLSv1_1       
          NET:SSLMethodTLSv1_2      
          NET:SSLMethodTLSv1_3      
          NET:SSLMethodTLS_MIN1_1  
          NET:SSLMethodTLS_MIN1_2   
          
          NET:SSLMethodTLS_MIN1_3  
        Hint: This property used to be called SSLMethod,
        and that name still works. Future releases may remove the name though. 
      Ciphers
       The term "TLS" covers a number of encryption schemes
        which may be implemented by both the server and the client. If the
        server and client can agree on a scheme, then the conversation goes
        ahead.
        
        You can test for the schemes supported by your server using the SSLScan
        tool.
        
        You can download a Windows version of SSLScan from 
http://code.google.com/p/sslscan-win/.
        Some documentation for the SSLScan tool can be found at 
          https://www.mankier.com/1/sslscan
        
        The two tests I recommend running are;
        
        
 sslscan --no-failed localhost:443
        
        Where localhost and 443 are the server, and port numbers respectively.
        This test shows all the Ciphers supported by your server.
        
        For a list of all the ciphers that SSLScan will test, along with the
        result, use
        
        
sslscan localhost:443
        
        The default cipher level will be  TLSv1.x High level ciphers only.
        
        
        It is possible to have complete control over the cipher list you
        support. the 
          ThisWebServer.SSLCertificateOptions.CiphersAllowed property is
        set to a 
Cipher List.
        
        This property is set in the 
WebServer procedure,
        in the 
INIT method, around priority 3000.
        It should come just after the generated line that sets the 
          ThisWebServer.SSLCertificateOptions.PrivateKeyFile property.
        
        The Cipher List string is a colon-separated list, where + means include,
        and ! means exclude. The format of the cipher list is documented at 
          http://www.openssl.org/docs/apps/ciphers.html#CIPHER_LIST_FORMAT .
        
        The default Cipher List (currently) looks like this;
        
          ThisWebServer.SSLCertificateOptions.CiphersAllowed = 
        
'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS!RSA'
        
        Note that this default changes from time to time depending on the
        current best practice. For the best security leave this setting alone,
        that way when the default is updated your program updates as well (the
        next time you compile.)
        
        From build 9.18 NetTalk supports ciphers which provide perfect forward
        security (ECDH). Not all clients support ECDH, and so those clients can
        fall back on the DH cipher. The DH cipher requires a file called
        dh2048.pem, which is in your application folder. You need to deploy this
        file with your application to get the DH cipher. 
        
        If the client does not support ECDH and you do not ship dh2048.pem, then
        they will fall back to using AES (which is still very good.)
        
        Here's an example alternate cipher list;
        
 ThisWebServer.SSLCertificateOptions.CiphersAllowed
          =
'DHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-GCM-SHA256'
        You may need to add more if you want to support older IE11 and Safari
        releases.
        
      
Port 80 Concerns
     Some developers and /or IT managers are reluctant to
      open port 80 to allow incoming traffic, perceiving it to be a security
      hazard. 
      
      This is an unfortunate "myth" that grew up a couple decades ago. At that
      time web servers could have vulnerabilities, and indeed the popular ones
      IIS and Apache did. Opening port 80 became a proxy for the problem – if
      your server was running on a different port, or if it was not exposed to
      the internet at all, then you were "safer". 
      
      Of course the problem is not the port number, it is the software running
      on that port. And today it’s so easy to scan all ports for all machines on
      the internet that the port number itself is not a mitigation anymore. 
[1]
      
      If you are running a NetTalk web server on a secure port (443) – then
      secure in this context has to do with the connection between the client
      and the server. Nothing more. It’s the software on the port that is the
      "problem" – not the port number itself, and not whether it is using HTTP
      or HTTPS. 
      
      So using a "Secure port" does not make the 
Server more secure,
      it makes the 
Traffic more secure. 
      
      In NetTalk the “insecure port” (assuming there is a secure port) is not
      served by the NetTalk Web Server. Rather it uses a simple class
      (_NetBabyWebServer) that doesn’t know how to do anything except respond
      with a redirection. (The exception to this is that it knows how to deliver
      the LetsEncrypt response.) Since it contains no code it cannot be
      manipulated to do unwelcome things. If anything this makes port 80 safer
      than the secure port 443, which is running the full featured web server
      class. 
      
      In summary it is not the port number which makes it a hazard, but rather
      the program running on the port. In NetTalk’s case this part of the
      program has no functionality, and simply issues a Redirect (301) reply. So
      there is no security hazard in play with this port open. 
      
      
[1] With the right tools scanning the internet
      can be measured in minutes. See 
https://thechief.io/c/editorial/how-to-scan-the-internet-in-5-minutes/Let's Encrypt DNS Challenge
     NetTalk 14 introduces the option of using the DNS
      Challenge offered by Let's Encrypt. While this approach is more
      complicated than the HTTP challenge, it does offer a useful alternative in
      specific conditions.
      
      Note: In both cases (HTTP or DNS challenge), the web
      server needs to make an out-going web-client connection to the LetsEncrypt
      server. This connection is done over port 443. It's the same as going to
      the 
LetsEncrypt website
      from a browser running on that server machine.
      
 Intranet Servers, not exposed to the
        Internet 
       The primary use case for DNS challenges are Intranet
        servers or servers running on machines that cannot listen on port 80.
        
        As discussed earlier, the HTTP challenge requires port 80 on the server
        to be open, and for the server to have a Internet IP address (ie a DNS
        name that resolves to a public IP address.) Intranet servers though
        cannot get a certificate this way. This results in the server getting a
        self-signed certificate, which then generates a scary warning when the
        site is accessed via a browser.
        
        The basic concept to solve this problem (first suggested by Jane
        Fleming) is to create a regular Domain Name, and to get a certificate
        for that domain, but have the domain name point to an Intranet IP
        address.
        
        The domain name itself is not important - it's just a name. It does not
        need to be related to the client. It DOES need to be registered at a DNS
        Provider that offers an API, and which is supported by a NetDNS class.
        (At the time of writing this is limited to 
DNSimple [1].)
        
        For example, we have a commercial product called EcoTime. So, at
        DNSimple we've registered 
EcoTimeLan.Com . 
        
        Now for each client we create a different subdomain (
something.EcoTimeLan.com),
        and set the IP address for that domain to the IP of the local Intranet
        server on the LAN. So, for example, at one of our customers (
hulk)
        our server is installed on a machine with the IP address of 192.168.2.11
        . So the domain 
hulk.ecotimelan.com points to 192.168.2.11 .
        If you go to that address in your browser, you obviously see nothing
        (unless you happen to have a server on your LAN on that specific IP
        address.) BUT, users on the LAN can simply put 
https://hulk.ecotimelan.com
        into their browser, and that translates to 192.168.2.11, and hence they
        get to our intranet server.
        
        So to be clear, the users on the customer site use your (provided to
        them) name to get to the server.
        
        So we have an intranet server, and we have installed our program on that
        server, AND we've created a DNS record for that server. The final part
        of the puzzle is to get the certificate. This is easy to do simply by
        setting the challenge type to DNS, and filling in some DNS API settings.
        (There's also a way to skip past the DNS API settings. see 
Getting
          a Certificate via an API)
        
        
 Note 1: The Link includes our affiliate ID. If you DO
        sign up, we get $5, and you get a $5 credit :), or just go to
        dnsimple.com ) ). 
Internet Servers that can't Listen on
        Port 80
       Another use case are Internet servers, but ones that
        can't manage HTTP challenges.
        
        If for some reason you have an Internet server, but access to the server
        over port 80 (to any program running on the server) is unavailable
          [2], then the DNS Challenge approach can also work. However since
        this requires that the domain be registered at a supported DNS provider
        (currently DNSimple) this is more work, and more expensive, to set up.
        That said, in some cases it may be advantageous to do so.
        
        
Note 2: Incidentally browsers still default to HTTP
        when the user types in a URL. If your NetTalk server is listening on
        port 80, then it is able to issue a redirect command to the browser, so
        that the browser switches to HTTPS. If you are not listening on port 80
        then the user will just get a "site unavailable" message - and they
        would need to know to change the URL to start with HTTPS. So a better
        approach is listening on Port 80. See above if you have 
Port
          80 concerns. 
Adding Support for DNS Challenge to a
        WebServer
      
        
          - Add a NetTalk Object Extension Template to the WebServer
            procedure. 
 Object name should be set to DnsClient.
 Base Class should be set to NetDnsDNSimple.
 On the Settings Tab Use In WebServer should be set
            ON.
- Make sure the Settings Control Template is populated on the
            WebServer window. 
 (You may need to re-populate this control template if upgrading from
            before NetTalk 14).
- Sign up to a supported DNS Service Provider. Currently DNSimple [3] 
- Register a domain there. (For example EcoTimeLan.Com -
            well, that's what we registered, you can't have that one.)
        Note 3:  This link includes our affiliate ID. If you DO sign up, we
        get $5, and you get a $5 credit :)  
 
      Deploying your Web Server - DNS
        Challenge
      
        
          - First Set up the DNS Domain, this is done outside your program;
 Go to the DNSimple dashboard in your browser.
 Add a subdomain for your domain, as an identifier to this server
            (for example hulk.ecotimelan.com).
 This can be an A (Ipv4) or AAAA(IPv6) entry (to match the protocol
            your server is using.)
 Set the IP address to the address of the server (LAN addresses are
            permissible.) For example hulk.EcoTimeLan.com, set to 192.168.2.11
- In the running server, go to the Settings / Certificates tab, and
            set the Challenge Method to DNS.
- In the running server, go to the Settings / DNS tab. Enter your
            DNSimple User ID and Password here.
- Once you have entered the User and Password  you can get the
            Account ID using the lookup button. 
 Or enter your Account ID if you know what it is.
- Set the Root Domain. 
 The Root Domain is the domain that you registered earlier (for
            example EcoTimeLan.Com)
- On the certificates tab, set the Domains list
            with the subdomain name (for example hulk.ecotimelan.com).
- On the certificates tab click the Check / Get
              Certificates button. (As with all deployments it's
            suggested you use the Testing Server until the process completes -
            then switch to the non-testing server once you have verified that
            the setup is all correct.)
 
    Getting a Certificate via an API
    Just when you thought that DNS challenges for Intranet
      servers was cool, along comes another option. This next approach is more
      complicated, and more expensive, but will appeal to developers who are
      rolling out many instances of their Intranet server. It is more work, and
      more cost, to set up, but makes deploying the Intranet server a bit
      easier. 
      
      One disadvantage of the DNS Challenge approach, as described above, is the
      use of the DNSimple User ID and Password being "known" by the Intranet
      Server. Although the password is stored securely, it is still exposed on
      the Intranet server, and in many situations this would be untenable. 
[1]
      
      A solution to this is to move the functionality for getting the
      Certificate into an API server. This means deploying your own API Server
      on the web. If you can do that, then the API server can interact with
      DNSimple (so your DNSimple ID and Password are not known by the client)
      and your Intranet Web Server program, running at all your customers, can
      interact with the API Server, not the DNSimple Server.
      
      Another advantage to this set up is that support and installation staff do
      not need to interact with the DNSimple Dashboard (step 1 in 
Deploying
        the Server - DNS Challenge), and hence do not need a DNSimple login.
      Your installation staff have access to the API Server (well, the web
      server part of the API server) and this reduces their task-list (and
      training) because the UI is tailored to the task they need to perform. For
      them it's a simple get-in, get-out.
      
      NetTalk 14 ships with an example API server, and an example Web Server
      that uses this approach. (
Examples\NetTalk\NetDns)
      
      
[1] The data does not 
need
      to be exposed to the user by making it a Run time setting. It could be
      compiled into the program, or the setting could be distributed as a
      pre-encrypted value, and simply read by the program.However even this may
      be untenable, because if these credentials leaked the whole domain would
      be exposed. Equally, if the credentials change, or you change domain
      provider, there would be a lot of installs needing updating.
      
 DNS API Server 
      
       Deploying the DNS API Server 
       This is covered in the documentation for the DNS API
        Server above. 
       API Client
       A stand-alone API client exists as a testing tool.
        It is located in \Examples\NetTalk\NetDns\apiClient
        .
      Adding support for API Client to your Web Server
      
        
          - Make sure the Settings Control Template is populated on the
            WebServer window. 
 (You may need to re-populate this control template if upgrading from
            before NetTalk 14).
 
       Deploying your Web Server
      
        
          - First Set up the DNS Domain, this is done outside your program;
 Go to the DNS API  Server in your browser (this is the DNS API
            Server you deployed earlier).
 Add a subdomain for your domain, as an identifier to this server
            (for example hulk.ecotimelan.com).
 Set the IP address to the address of the server (LAN addresses are
            permissible.) For example hulk.EcoTimeLan.com, set to 192.168.2.11
- In the running server at the customer, go to the Settings /
            Certificates tab, and set the Challenge Method to
            API.
- In the running server at the customer, go to the Settings / API
            Challenge tab. Enter the API Server URL to your
            DNS API Server here. 
 Also set your DNS API User ID and Password
              here.
- On the certificates tab, set the Domains list
            with the subdomain name (for example hulk.ecotimelan.com).
- On the certificates tab click the Check / Get
              Certificates button. (As with all deployments it's
            suggested you use the Testing Server until the process completes -
            then switch to the non-testing server once you have verified that
            the setup is all correct.)
 
    Troubleshooting and Common Errors
     The first thing to do if there is a problem with the
      secure server is to go to the WebServer procedure, to the NetTalk
      extension (for the server) and turn off the option
 Suppress Errors.
      This will usually quickly highlight what the error is. Once the error has
      been corrected you should turn Suppress Errors back on.
      
      Some common errors are;
      
-65: SSL Failed to Load Certificate
       Secure sites need a .CRT
        and a .KEY file. This is usually in the app\certificates folder. If one of these files
        are missing you'll get this error.
      -73: NetTalk Could Not Load TLS DLLs [LibCrypto-1_1.DLL,
        LibSsl-1_1.DLL]
       The TLS functionality in NetTalk requires the 
        
LibCrypto-1_1.DLL and 
LibSsl-1_1.DLL
        files. These can be found in your application folder or in the 
\clarion\accessory\bin
        folder. Copy these two DLL's to your application folder (and include
        them in your program install.)
        You also need to ship 
CAROOT.PEM and 
DH2048.PEM.
        
        You may also need to install the 
Visual Studio 2017 (x86) runtime on the target
        machine. On most desktop machines this will be installed already, but on
        Server machines you will most-likely need to install it.
        
      
Common Errors when getting a LetsEncrypt certificate
       When getting certificates from the LetsEncrypt
        server the process is logged in the text control on the window. This
        helps you to see where it might be going wrong. Here are some common
        messages when the process does not work.
        
VCRuntime140.DLL Missing
        
        Unable to get certificate -
          Challenge was invalid
         The process of getting a certificate from LE
          involves your server placing a file on the disk. Then LetsEncrypt will
          call your server, using the normal web request, to get the file. This
          is how you prove that you are in control of the actual server at that
          domain.  This is why this process has to run on the actual real
          server. It can't run on a development machine and succeed.
          
          If you get the error above it means that the LE server failed to get
          the file. 
          
          The log of the process looks something like this (read this from
          bottom to top);
          
          
5. Unable to get certificate - Challenge was
            invalid
            4. Checking Status
            3. Notify Server Challenge is Ready
            2. LE Server will now fetch
http://www.somewhere.com:80/.well-known/acme-challenge/C4cGfKzgY3B-EWqSKuKgPBH0
            1. Challenge Token Saved
C:\somewhere\web\.well-known\acme-challenge\C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
          
          Debugging this process is straight-forward, because you can check each
          step. Starting from the bottom;
          
          
1. Challenge Token Saved
           C:\somewhere\web\.well-known\acme-challenge\C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
          
          Make sure this file has been created, and it exists. It will contain
          two hashes, separated be a period. For example;
          
70NW1-dB3MVucYhV3E0ExrUy3pPD7ca8ZKbHO3nGtVs.Qd6eZFsIb1jZt6orgpYEelEz-rmHppspMi_SCPK8Br0
          
          This file should be in a place where the web server listening on port
          80 (either your server, or some other server) can serve it from. If
          the location is not correct you can adjust it by setting the 
AcmeWebFolder setting.
          
          
2. LE Server will now fetch 
           
http://www.somewhere.com:80/.well-known/acme-challenge/C4cGfKJruOS11cSL8GxL5g97nzgY3B-EWqSKuKgPBH0
          
          Go to your browser (preferably on a machine not the server) and enter
          this URL. If the server is set up correctly then you should see the
          contents of the file in your browser. If you do not see the file, then
          you need to revisit step 1, and also check your server to make sure
          it's listening on port 80. If LE can't fetch this file it will not
          create the certificate for you.
          
          
Hint: If you are using IIS on Port 80,
          then it may need to be configured to server files with no extension.
          This is done in the 
web.config file -
          for example;
          
          
<?xml version="1.0" encoding="UTF-8"?>
            <configuration>
               <system.webServer>
                    <staticContent>
                        <clear />
                        <mimeMap
            fileExtension=".*" mimeType="text/json" />
                    </staticContent>
                    <handlers>
                        <clear />
                        <add name="StaticFile"
            path="*" verb="*" type=""
            modules="StaticFileModule,DefaultDocumentModule,DirectoryListingModule"
            scriptProcessor="" resourceType="Either" requireAccess="Read"
            allowPathInfo="false" preCondition="" responseBufferLimit="4194304"
            />
                    </handlers>
                </system.webServer>
            
            </configuration>
          
        [End of this document]