CapeSoft.Com
Clarion Accessories
NetTalk
Doc Index
Apps
CapeSoft Logo

CapeSoft NetTalk
Apps

Download Latest Version
Installed Version Latest Version

NetTalk Apps

Introduction

NetTalk Apps is the third level of NetTalk. It includes all the functionality in NetTalk Desktop and NetTalk Server. In addition to that it also allows you to create disconnected web applications and native mobile applications for use on phones and tablets.

To best understand what NetTalk Apps is doing, it's first necessary to understand a simple premise you have used up to now, and what the limits of that premise is.

When writing Clarion Desktop (aka Win 32) programs it is understood that there is one data store. Multiple users can access that data store, but fundamentally there is a single data store which all users are sharing. This premise still exists when writing a NetTalk Web application. Many users sharing one exe, which in turn is sharing one store.

In this context a data store can contain multiple databases, it may be ISAM (TPS) or SQL based and it may encompass multiple folders on a disk. But at it's heart the idea is that data is stored in a single place and (ideally) you don't have the same data stored in multiple places.

Of course data does occasionally need to be replicated into multiple places, and maintained in multiple databases automatically. This can be done via replication - either Server-side replication (which requires that all databases be of the same SQL flavor) or client-side replication (as with the CapeSoft Replicate extension) which requires that all programs be written in Clarion.

With NetTalk Apps the goal is to allow different programs with different data stores, written in different languages, to share data in such a way that the data can easily flow between many different stores, and importantly, between many different kinds of stores. The actual storage of the data, and the capabilities of that storage vessel, cannot be taken for granted. Data might just as easily reside in an XML file, a SQL database, a browser's Local Storage or a local ISAM file. The goal is to allow many different kinds of programs to interact with their own data, and then share that data with each other.

In some ways this means thinking "beyond SQL". SQL has traditionally been the store of choice for many programs, and indeed there are many benefits to having SQL as a store, but data inside a SQL database requires clients to connect to that database. In order to create disconnected programs we have to allow the data to escape the SQL database, returning (possibly changed) at some later point. Of course this does not replace the SQL database - your primary store will likely remain as a SQL database (although it could also be TPS) - but rather it expands the horizons of the database.

Distributed Data Synchronization

Introduction

Before you can start building disconnected apps you need to understand the concept of distributed data. By definition disconnected apps keep a copy of the data, and that copy has to be synchronized with the parent server from time to time. In order for that to work correctly the tables need to contain specific fields and the records need to be updated in specific ways.

It should be noted that the mechanism described here is specifically designed to be completely independent of the data store. If all the databases were in the same store (like say MS SQL Server) then it would be possible to use the replication built into that server to do the necessary synchronization. However as the data will be distributed to many different platforms (Windows, Web, iOS, Android etc) it is not possible to ensure that only a single data store is used. Equally this approach is not limited to any one programming language. NetTalk contains implementations in Clarion and JavaScript, but the system is language agnostic and so any program written in any language can join in, as long as it obeys some rules.

Summary

The logic behind these requirements are discussed below, but this is the checklist of requirements (for a Clarion app):
  1. Each table needs a GUID field - type String(16)
  2. Each table needs a GuidKey, marked as unique, on the GUID field.
  3. Each table needs three TimeStamp fields, all of type Real, TimeStamp, ServerTimeStamp and DeletedTimeStamp. These should have external names of ts, sts and dts respectively.
  4. Each table needs a ServerTimeStampKey key on the ServerTimeStamp field. The key is not unique.
  5. Each table needs a TimeStampKey on the TimeStamp field. The key is not unique.
  6. Whenever a record is updated, TimeStamp must be set to the current time stamp. In Clarion apps this is usually done with the NetTalkClientSync global extension template.
  7. Whenever a record is updated on the parent server, ServerTimeStamp must be set to the current time stamp.
Remember the GUID field has two strict rules.
  1. The contents of the field must NEVER be changed.
  2. Any attempt to populate it using other data should be avoided. It should contain pure randomized characters.

Dictionary Fields

GUID Field

This field is required. It is a unique row identifier. This is (arbitrarily) a 16 character string using a 36 letter alphabet.

Although some databases have a native data type named Guid this field should not be native to any specific database. Rather it is a simple 16 character random string. To make the field both portable to multiple databases, and easily transportable between databases, it is recommended that an alphabet of 36 characters (a-z, 0-9) be used.

If considering additional characters in the alphabet:
  1. Characters > character 127 would render the value not utf-8 (which has transport implications)
  2. Any punctuation may be a problem with current or future transport or encoding situations. For example, “ and \ characters in JSON, < and > in xml and & in HTML
  3. Whitespace characters (especially tab, cr and lf) may be incompatible with file formats
  4. Any control characters (ASCII 0 through 31) should be avoided
  5. Some systems are case sensitive, others are not, hence either upper, or lower case letters are preferable
Usually this field forms the primary key for the table if this is a new table. If another primary key exists then that's ok, but this field must be unique in the table. Foreign keys in other tables can use this as the linking key field.

For example:

Invoices Table
Inv:Guid

Line Items Table
Lin:Guid !Each entry in this table has its own unique Guid value
Lin:InvoiceGuid !Refers back to the invoice this line item belongs to

The Guid field contents cannot be changed at any time for any reason.
This avoids the complexity and performance involved in doing related table changes. It is also necessary for this field to be unchanging in order to track a specific row through the (disconnected) data stores. If the Guid field changes, then all other databases are immediately inconsistent and a consistent state cannot be achieved.

Any attempt to place order on the Guid field, by populating it with some calculated rather than random field, should be avoided.
Embedding data in the Guid could lead to that data either becoming inaccurate, or needing to be changed. Additional fields should be added to the row for storage of additional data. The Guid should contain completely random values.

Mathematically a 16 character string using an alphabet of 36 characters allows for 36 ^ 16 or 7 958 661 109 946 400 884 391 936 possible identifiers. This can be written as 7.9 * 1024. There are approximately (very approximately) 7.5 * 1018 grains of sand on earth. So a 16 character Guid is sufficient to identify, uniquely, all the grains of sand on 100 000 planets similar to Earth. Mathematically, yes, a collision within a single table is possible, however it is not probable.

TimeStamp field

This field is required for all tables that allow updates in any non-root database. A Real allows for 15 significant digits. This is sufficient to hold the number of milliseconds since 1970, which is the format used by JavaScript.

ServerTimeStamp field

This field is required. It is a Real holding the timestamp as last set in the server. On the server this value is always equal to Timestamp. On the mobile device this value may be different to Timestamp if the record has been updated on the device. (The device holds the time of the last update on the device.)

DeletedTimeStamp field

Acts as a deleteflag, and contains the time when the record was deleted. 

CreatedBy field

This field is optional, but can be useful when distributing a limited data set to other devices. The type is up to you, but should match an identifier used by your existing users system. Assuming you have a Users table, and that table has a GUID identifier, then this field would also be a String(16).

UpdatedBy field

This field is optional, but can be useful when distributing a limited data set to other devices. The type is up to you, but should match an identifier used by your existing users system. Assuming you have a Users table, and that table has a GUID identifier, then this field would also be a String(16).

Current Time Stamp

While time stamps are necessary in order to compare records, and choose the later record, the actual measurement of the time is very arbitrary. In order to provide a generic system, that can be used across many platforms and programming languages, Unix Epoch time is used. However Unix Epoch time is typically measured in seconds, whereas NetTalk allows for milliseconds. If sub-second timings are not required, or available, then multiply the Epoch time by 1000.

Data is independent of time zone, therefore all time stamps are relative to UTC regardless of the local time zone where the data is added, updated or deleted..

The data type used will vary depending on the platform and language. A type which can contain integer numbers in the range 0-4 000 000 000 000 should be tolerated. (This allows for the system to work until at least 2095.) In Clarion the best type for this is a REAL which easily exceeds this range.

All NetTalk objects expose a method, GetElapsedTimeUTC(), which returns the current time stamp (measured as the number of milliseconds since Jan 1, 1970.)

Here are some examples;

Clarion Dictionary Tips

  1. Add a Globals data section
  2. Add a variable, st, to this section. Set the type to STRINGTHEORY
  3. Start with one table, and add all the fields (GUID, TimeStamp etc). You can cut and paste all the fields from this table to the other tables, so it's worth getting them perfect before doing that.
  4. Set the Guid field, Initial Value, to Glo:st.MakeGuid()
  5. For the timestamp fields be sure to set the external names (to ts, sts and dts).
  6. For the Guid and all the TimeStamp fields go to the Options tab and tick on Do Not Populate.
  7. For the TimeStamp field go to the Options tab and enter an Option called TimeStamp, with the value 1. Do the same for the ServerTimeStamp and DeletedTimeStamp fields, setting their options to ServerTimeStamp and DeletedTimeStamp respectively.
  8. Add a Key on the Guid field, a key on the TimeStamp field, and a key on the ServerTimeStamp field to the table. (At the time of writing it was not possible to cut & paste keys between tables, but future versions of Clarion may be able to do that so test to see if it does.)

Database Field Constraints

It is common in data design to create constraints on data where one field "must exist" in another table. For example in a LineItems table you may place a constraint that the InvoiceGuid field (which identifies the parent invoice record) "Must be in File" for the Invoice table. Thus creating an explicit constraint that the parent record (Invoice) has to be added to the database before the child record in the LineItems table can be added.

Data distribution as described in this document does not make this constraint impossible, but having this constraint does make the system more complex and leads to other potential problems. Specifically the order of table synchronization becomes important, and each table has to be completely synchronized before another table can be synchronized.

As an aside, in a Clarion application (regardless of backend), the "Must be in File" constraint slows down the ADD to a child table enormously, so it's a very expensive constraint there. In a SQL application the constraint is a lot less costly, but will cause the problems mentioned above. For this reason it is recommended that this constraint be removed from systems which support data distribution.

Consistency

Consistency is the concept that all the instances of the database have the same data.

A disconnected / synchronization system, is by nature not always consistent. There are times when the mobile device is disconnected from the network and so has a possibly old version of the database. Since the program is reading from the local data store the value returned is the last known correct value, but this may differ from a value elsewhere in the network.

The system however is eventually consistent. Meaning that once all data stores have synchronized with the server, they will ultimately end up with the same version of the data.

Transactions

Consider for a moment the nature of a transaction. It provides an atomic wrapper around multiple writes to the database, ensuring that if any write is made, all writes are made and if all writes cannot be made then none is made.

The key phrase in that sentence is the - as in the database. How then do transactions occur when the database you are writing to is a local data store, and not necessarily the main data store? Log based replication systems are able to support transactions in the sense that the transaction frame can be written into the log file.

However a disconnected system, such as the one described above, synchronizes the end result, on a table basis, and does not replay individual writes, or maintain the order of the writes.

There are fundamentally two kinds of transactions. Insert transactions are not a problem. They are added to the local store using whatever mechanism the local store offers. They can be in one, or multiple, tables. When synchronizing all the records will be written to the server.

Edit transactions are more problematic. Consider a common case;
When selling stock of an item, the quantity of the item is set in the Invoice Line Item, and deducted from the inStock value held in the Products table.
In this case the inStock value is a running total and every edit or insert in InvoiceLineItem requires a balancing edit to this value.

This sort of transaction cannot be done in a distributed system, and indeed running total fields (or calculated fields) of this nature cannot be maintained using simple data syncronization.

One approach to solving this problem is not to write to the running total at all at the deice, but rather set the server to update this total as it updates the lineItems table. In other words move the maintainence of the running table to the server rather than doing it on the device.

Another approach is to calculate running totals, rather than storing them. So, rather than Edit records to indicate a change, Insert a record to a table so that the change can be recalculated at will.

Auto Numbering

Auto Numbering is a common approach to generating a unique ID field for each table in a dictionary. In most cases the auto numbered value is a surrogate value (ie does not correspond to any actual value in real life - it's just used as a unique ID) although in some cases it may be a natural value (which has meaning, for example an Invoice Number.)

Auto Numbered values are usually sequential integers - as each record is added the next ID is fetched from the server and used as the ID for that record.

Auto Numbering in order to create a unique identifier has a number of disadvantages, not least of which is the requirement that there is "one entity allocating numbers". In a distributed system it therefore cannot be used as a means to generating primary keys values, since multiple (distributed) systems can create records at the same time.

Auto Number can still be used for natural values (like invoices) with the proviso that they can only be allocated when the data reaches the (single) parent store. So multiple systems could add Invoices (using the GUID as the primary key) but the actual Invoice  Number is only allocated when the data is synced with the parent store.

Of course since the Auto Numbered value is a natural value, it should not be used as the linking field to any child tables. The GUID of the parent table should be used as the linking value.

Common Requirements

The majority of requirements for disconnected applications are the same regardless of whether you are doing disconnected desktop, web or mobile applications.

Requirements

The following global templates are required;

Field Rules

The fields in dictionary need to be set as follows;
  1. GUID field
    Set to a Random 16 character string when a record is created
  2. TimeStamp field
    Set to the current time stamp when a record is created, updated, or "deleted".
  3. ServerTimeStamp field
    Unaltered in the desktop program. In the server program (and ONLY the server program) is set to the current time stamp.
  4. DeletedTimeStamp field
    If a record is "deleted" by a user then the DeletedTimeStamp is set to the current time stamp. The record is NOT deleted from the table, it is written to the table as an update. Browses (Reports, Processes etc) in the table would need to be changed so that records marked as Deleted are not included in the Browse or Report etc.

Deleted Records

Most of the changes required to turn a "normal" Clarion ABC application into disconnected desktop application can be automated. The necessary changes to the timestamp fields and the changing of a Delete into an Update can be managed by a global template overriding the File Manager object.

However procedures that read the tables will need to be adapted. The tables will now contain records which are marked as deleted, and these records need to be filtered out from browses, reports, processes and so on where appropriate.

In the case of browses it may be desirable to offer the user a switch to show, or hide, deleted records - ultimately allowing them to be undeleted (by setting the deleted time stamp back to 0.)

Unique keys, other than the GuidKey, also need to be considered in the context of deleted records. For example, if there is a unique key on the Name column, and a Customer "Bruce" is deleted, then adding another customer (or the same customer) back with the same name will fail, even though no customer called "Bruce" appears to be in the table. Adding the DeletedTimeStamp as an additional component to these unique keys may suffice as a solution.

Template Support (ABC)

A Global Extension template, called NetTalkClientSync template can be added to ABC applications. If you are making a disconnected desktop application then you will need to add this template to the desktop app. If you are creating a disconnected web, or mobile, application then add this template to the web app. While you do not need to add it to the SyncServer app, it does no harm to have it in that app, if that app is shared with say the web app.

This template performs the following chores;
  1. A global utility object called glo:SyncDesktop is added to the application. This object can be used to get the current time stamp. As in;
    glo:SyncDesktop.GetElapsedTimeUTC()
  2. All the time stamp fields are updated appropriately when records are inserted or updated in any table. This is done in the FileManager object, so will apply as long as ACCESS:table.INSERT method is used for adding records (not just a file driver ADD or APPEND) and ACCESS:table.UPDATE is used instead of the file driver PUT command.
  3. When a record is deleted, the DeletedTimeStamp is set, and the record is updated, not deleted. This works as long as the ACCESS:table.DELETERECORD method is used and not the file driver DELETE command.

Apps

Conceptually in any disconnected system there are at least two parts which need to work together, the client program and the server program.

In the case of a disconnected desktop system this usually means two separate apps, a desktop app, and a server app.

In the case of a web (or mobile) program though the server app can fulfill both the server role, and the client role. In other words in the web case the server part and the client part can be developed in the same app file.

Server App

The server app consists of a single NetWebService (usually called Sync) and a number of NetWebServiceMethods. Usually there is a single method for each table in the database. This application can be generated for you by the NetTalk Wizard. For more information on generating this application see the section Server for Disconnected Apps below.

You can then expand the Server app to include a web-client interface. This interface can be a normal web app, or a disconnected web app. If a disconnected web app then it can be packaged as a stand-alone mobile app as well.

Non-Clarion (or PROP:SQL) Writes

If the data in the table is edited from a program not using the NetTalk Client Sync Template, then the rules for updating the Guid and Timestamp fields MUST be observed. This includes programs like TopScan, SQL Manager, and PROP:SQL statements that do direct writes to the database.

Use of this distributed data system specifically does not limit you to Clarion programs. Any program in any language can read and write to the data, using any appropriate technique, as long as the field rules are followed. Some samples of getting the time-stamp value in various languages are;

Server for Disconnected Apps

Introduction

Creating the server for a disconnected app is very straight-forward because it's either completely automatic, or only minor changes to the wizarded app is required.

Creating using the Wizard

  1. Create a new application as normal


  2. Select the Dictionary file, and make sure the Application Wizard option is on


  3. Select the Server App Wizard from the list


  4. Click Next on the Wizard Welcome screen
  5. On the What Do You Want to Create tab select Server API's to Sync Disconnected Data


  6. On the Global Extensions tab select all the Required options (except for OddJob). Add the Optional extensions as desired.


  7. On the Procedures tab set the items as shown


  8. Select the Files to Sync. This will usually be all of them, or almost all of them.


  9.  A Sync server is still a web server, and documentation for the API's is automatically generated. Select your preferred Application Theme Option. Click Next.
  10. Select your preferred menu. Click Next.
  11. Set the server options to not-secure, or secure. Click Next.
  12. Set Header and Footer options. Click Next.
  13. Set IndexPage options. Click Next.
  14. Set the sync Service name. The default value is sync.


  15. Click Finish to complete the Wizard.
  16. You will likely see a message along the lines of the picture below. Click on Yes.

  17. Your app will be created. This application is set up to be your Disconnected App Server. The only additional work will be normal new-app things, like setting variables to connect to the database (if SQL) and so on.

Adding Support for New Tables

If you have an existing Sync app, and you add a new table to your dictionary then you will need to add a method for that table to your server app. To do this; Click on t
  1. Click on the New Procedure button
  2. Enter a name for the new procedure. The standard is to use the prefix sync and then the table name. This convention is necessary to make it easy for the client application to call the right procedure.


  3. From the list of Wizards choose the ServerWebServiceMethodWizard. Then click on the Select button.


  4. On the Method Procedure Options screen make sure the This is a Sync Method checkbox is on. Also sect the table that this method will sync.


  5. Click on the Finish button to complete the wizard.

Disconnected Desktop Apps

Introduction

Disconnected Desktop apps basically have a local copy of the data. The program itself is written to talk to this local copy pretty much like any Clarion desktop program would talk to any database.

Then a "Sync" procedure is added to the local program. This procedure synchronizes the local data with the remote NetTalk API "Sync" Server. That Sync Server then interacts with the server-side database.


This architecture is useful when the connection between the desktop program is either too slow, or too unreliable for direct connections to the main server in the cloud.

The key is in noticing that the desktop program, and the desktop data is completely unconcerned with the network connection. It is the responsibility of the sync procedure, which is running in a separate thread, or even a separate process, to communicate changes with the parent server, and to get changes made in the parent server back to the desktop.

This architecture is not limited to a single desktop data set. The following is also valid


Multiple different locations can synchronize with the same server.

Sync with Server

A Sync procedure can be imported from the example desktop.app application. This example is in \clarion\examples\nettalk\
In order for data to flow between the desktop app and the server, a synchronization procedure is required. This procedure can be a background procedure in your application (ideal where a single user exists for the remote program) or as a separate program on the LAN. The sync procedure can be triggered by a timer (every few minutes or so) or it could be triggered by NetRefresh so that it synchronizes after every write.

If the desktop is unable to connect to the server then the desktop will continue working as normal, and the data will then be sync'd on the next working connection.

The Sync procedure is created as a simple window with the NetTalkClientSyncControls Control template added to it. This template requires the NetTalkClientSync global extension be added to the application.

The Sync procedure runs on its own thread and can be communicated with using events. The thread number of the procedure is stored in glo:SyncDesktop.ControllerThread

A Sync can be triggered at any point, by any thread, by simply posting Event:SyncNow to the Sync thread. If a Sync is currently underway then it will complete the current sync and do another full sync immediately afterwards. For example;

Post(Event:SyncNow, ,glo:SyncDesktop.ControllerThread)

The Sync itself is a fairly cheap operation with minimal overhead, so it can be called on a regular basis. If not data in a table has been updated on the server, or client, then it's a very small request and response for the server.

Note  the Sync Procedure will not create a table on the desktop app if it does not already exist. If you want the sync procedure to create the table, then add the table to the data pad for the procedure.


Disconnected Web Apps

Introduction

A disconnected web app is a web application where the browser can be disconnected from the network, but the web app itself continues to work. In other words a user can go to the web site (say via a Wi-Fi connection) start working, then walk away, out of Wi-Fi range. While out of range the app continues to work, and the user can continue to capture and edit data. Once the user is back in range, and connected to the server again, then the data is automatically synchronized.

The basic architecture for a disconnected web app is very similar to a normal web app, however there are some key differences;

It should be noted that a web app can contain a mix of connected, and disconnected areas. In other words a single system can contain both a "normal" web app, as well as one or more "disconnected" apps. This leads to some flexibility in the way the application is designed.

Supported Browsers

Disconnected Web Apps make use of the HTML 5 feature called "Local Storage". This allows browsers to store information in a data store belonging to the browser, which in turn means the app can work when it is not connected to the network. Local Storage is supported in all the modern browsers with the notable exception of Opera Mini.

In iOS 5 and iOS 6 it's possible for data in the local storage to be cleared by the OS, so use of devices running those operating systems is not recommended.

IE7 and earlier is not supported. Use of any version of IE for disconnected web apps is not recommended as Local Storage under IE has a number of different behaviors compared to other browsers.

Task Based Design

Because of the above differences it is very important that the design of the application be focused around what tasks the user will need to accomplish when using this app. It is probable that in many cases only a limited sub-section of functionality is required by a user when they are disconnected from the network. By clearly thinking through the tasks which you need to support, you will better be able to design the appropriate application.

Summary

  1. Global Extensions, Activate NetTalk Web Server extension, Advanced tab. Tick on;
    Generate for Disconnected App is on.
  2. Make sure the menu is set so that all the links are opened as Popup. Links set as Link or Content are allowed, but those links will not work if the browser is disconnected from the network (ie cannot access the server.)

Application Cache

The Application Cache Manifest file, while not absolutely required in order to do web apps, is a way of explicitly telling the browser which resources will be required by the app. This feature is supported all major browsers except for IE9 and earlier, and Opera Mini.

The application cache file is generated into the web folder as app.appcache where app is replaced by the name of your application. A Manifest attribute in the HTML of your root page tells the browser of the appcache file.

You do not need to do anything to turn this support on.

Native Mobile Apps

Introduction

Because NetTalk now creates disconnected web applications, it is possible to take those web applications and repackage them into a native app format.

For example, using Adobe PhoneGap (which in turn is built on Apache Cordova), it is possible to wrap the HTML, CSS and JavaScript into a native application which then uses a native HTML control to show the HTML to the user. This application contains a mix of native code (eg ObjectC on iOS or Java on Android) and the HTML you have created. In addition frameworks like PhoneGap extend the JavaScript language to give the JavaScript engine access to the native operating system API, which in turn means you have access to the hardware, contacts list and so on.

Apps built in this way are sometimes know as Hybrid apps, because they combine the portability of HTML and JavaScript, with a minimal amount of generic native code to expose the underlying hardware and OS.

The big advantage of this approach is that multiple operating systems can be supported from a single code base. This makes it ideal for systems which need a mobile application, but which don't necessarily have the economic viability of re-coding the same app multiple times (especially for some of the more minor platforms.)

There are two disadvantages with this approach though. The first is raw performance. Because the code is HTML and JavaScript it does not have the raw performance of a native ObjectC or Java program. This makes it unsuitable for high-performance situations, like games. For data collection applications though performance is certainly good enough, and the responses are mostly quite snappy. The second disadvantage is that the interface can look, and behave differently to a native application. Using themes and good layout it is possible to mimic functionality to a high degree, but it will never be 100% the same.

Hybrid apps using PhoneGap can be added to the Apple AppStore but must still go through the Apple approval process just as any iOS app needs to do. As such it must conform to Apple's standards for design and usability. Apps can also be submitted to the Google PlayStore (which has a much lower set of standards and requirements.)

NetTalk generates the necessary file to make it easy to package the application using Adobe PhoneGap.

Adobe PhoneGap

PhoneGap is an open source framework, currently owned by Adobe, which makes it easy to package hybrid applications. PhoneGap was released under an open source license as Apache Cordova. the current PhoneGap is an Adobe product, built on top of Cordova, which encompasses some extra tools.

PhoneGap supports a number of platforms including iOS, Android, Windows Phone, Blackberry, Bada, Symbian, webOS, Tizen, Ubuntu Touch and Firefox OS.

Adobe PhoneGap Build

PhoneGap Build is an online platform that allows you to upload all your HTML, CSS and JavaScript and then does the heavy-lifting of turning that into a native iOS package or Android SDK. You don't have to use PhoneGap build, but it is certainly a convenient and reasonably cheap way to get started. You do not have to use PhoneGap Build, it is perfectly possible to create your own build environments, but it is a recommended way to start.

PhoneGap Build supports iOS, Android and Windows 8 as target platforms.

PhoneGap Build requires a configuration file, called config.xml, which contains information for the build system. This file is generated for you by mBuild.

NetTalk PhoneGap

Using NetTalk to generate Mobile applications using PhoneGap is the result of many NetTalk technologies working together.

mBuild

mBuild is a utility that ships with NetTalk Apps. It does the work of turning your disconnected web application into an APK, IPA or XAP file. For more on mBuild see the mBuild documentation here.

Techniques for Disconnected Applications

This section covers areas where programming a disconnected (browser) application differs from a connected web application.

Custom JavaScript File

In a web application it is unnecessary for you to write custom JavaScript functions as the server is able to let you write your code in Clarion. In a disconnected application though all the code has to be written in JavaScript. While the templates will generate most of what you need, you will almost certainly get to a place where you want to make some small adjustment or add some code not currently generated by the templates.

To do this you will need to create a custom JavaScript file, and link that file into your application. Once that is done you are free to add to the file whenever you need to.

JavaScript files are simple text files, usually with the extension .js. They belong in your \web\scripts folder. You can name the file anything you like - something unique and related to the application is recommended.

Once you have created the file, add it to the WebServer procedure. It is added on the NetTalk extension, Settings / Scripts tab, in the scripts list. For now just add the new file to the list, and accept all the default settings for the file.

Single Record Settings Table

In a web app there are multiple users using the same server, and so there is typically a record in a user table for each user. This table contains all the settings for a user.

By contrast, in a disconnected app it is often necessary to have a "settings" table for the user using that device. This is a single-record table, and contains important settings which allow the app to work.

Specifically it will likely contain one or more of the URL of the Sync Server machine, the User name and Password. Since it contains information needed to connect to the sync server, the Settings form needs to be functional without a connection to the server.

The application needs to be able to do the following;
  1. Add a single record to the table if it does not already exist.
  2. Prime the GUID field in the new record to a unique value.
  3. Open a form to allow the user to edit the fields in this record (although the GUID of the record is not known at the time when the program is generated.)

Fortunately there is template support for a single record table, and this generates the necessary JavaScript to add the first record for you. The settings for this is on the Global Extensions, the "second" NetTalk global extension (Activate NetTalk Web Server), on the Apps / Settings Table tab. Enter the settings table here, as well as identify some common fields that might be in your table.

The above template settings take care of the first two requirements in the list. The last one is done when you open the form. Because the GUID of the row is unknown, the unique id value can be set to the special value, _first_. The form will then understand that you mean to edit the first row (in this case the only row) in the table.

User Authentication

One of the aspects of App development you will come across fairly early is identifying, and authenticating, the user who is using the app. Typically you will want to restrict access to the Web API to users using your app, or you may want to restrict the data being sent to the user based on who they are.

There are many ways to approach this, this document shows some possible methods, but it doesn't cover all the possibilities. Feel free to salt to taste as required.

The simplest approach is to make use of a Single-Record settings table, as described above. Make sure the table has a field to hold the user name and another field for the password. Then assign those fields in the global description.

If this is done then those fields are automatically used when making a sync request to the server. They are used to construct the HTTP Authentication header, using Basic Authentication. If done over a TLS (SSL) connection this is safe and secure.

On the server side, the authentication needs to be validated in the WebHandler procedure, in the Authenticate method. The user name and password will be passed in there and you can authenticate it there.

Data Type Selection and Transformation

It is common when designing a database to distinguish between the display of the data and the storage of the data. For example a date is usually displayed as a string (3 Oct 2016 or 10/03/2016) but stored as a DATE or LONG type.

In order for this to work data has to be transformed when read from the database before being displayed, and after user entry, it has to be transformed again before being stored in the database. In Clarion this happens automatically, based largely on settings set in the dictionary. Since other languages do not have a dictionary this process becomes very manual, and has to be applied manually on a field by field basis.

For this reason it is suggested that the storage, and display, of the data be the same - at least at the mobile app end of things. The server side can then transform the data as required between a string and a local data type if desired.

This process is automated by the templates for the DATE and TIME field types, as well as for LONG fields set in the dictionary with a @D or @T picture. In these cases the field generated in the JavaScript is a String, not a number. For the Sync methods on the server the incoming data is automatically transformed back into the appropriate numeric (DATE, TIME, LONG) format.

Priming Fields on a Form

The normal NetWebForm template has a tab for priming field values. This can be used to prime the fields on a form when the form opens. The priming can be dependent on the action, for example a date field may be set to TODAY() when a record is Inserted into the table.

In a web app the priming happens on the server side, and so Clarion code can be used. In a disconnected mobile app, the priming happens on the client side, and so must be set to valid JavaScript code. In the template settings on the Priming window, you can set both Clarion code, and JavaScript code to do the priming.

Special JavaScript functions are available to make this task a little easier. Remember that all JavaScript code is Case Sensitive.
CodeUse
today()Primes the field with today's date, in mm/dd/yyyy format.
today(pic)Primes the field with today's date, in the selected picture format. Currently available pictures are @D1, @D2, @D5, @D6, @D9, @D10. Note that all pictures with yy are formatted with yyyy. For example;
today('@D6')
clock()Primes the field with today's time in hh:mm:ss format.
Note also that the From Location sub-tab (on the Priming tab) can be used to prime fields with the current Latitude and Longitude. Fields to receive these values need to be on the firm, but they can be set as Hidden fields if necessary.

JavaScript

A general discussion of the JavaScript language is beyond the scope of this documentation, however there are some useful tips around the specific NetTalk JavaScript which may be useful here.

Database.js

Note: As from build 9.18 the resultset property has been removed. Functions using resultset (idbSelect, idbSummary) should pass in a resultset parameter array instead.

The tables from your dictionary, and a number of helper functions, are generated into scripts/database.js. This file contains the table structures, as well as other information for each table. It also contains a number of properties for the database itself.

A global object called database is declared.
This contains a number of properties (such as synchost, user, password, and so on.) You can use these properties in your own code if you need to. For example;
var user = database.user;

It also contains an array of tables. One item in the array for each table used by your application. As it is an array each table has a number, but this number could potentially change if new tables are added to the system. Each table also has a name, and you can reference a table using this if you like. For example;
var sometable = database.tables[0]
JavaScript arrays are base 0.

Here's an example of code to loop through all the tables;

var j = 0;
for (j in database.tables){
    if(database.tables[j].name == 'customer'){
        sometable = database.tables[j];
    }
}

To make things a bit easier an alternate name is generated for you. Instead of referring to the table by its number in the array, you can refer to it by name. For example, for a table called customers;

sometable = database.customers.table;

As you can see this is the simplest approach if you are accessing a single table with a known name.

Each table has a number of properties (name, keys, relations and so on) but importantly it also has a record. This contains the individual fields that make up the data area of the table. For example;

somename = database.customers.table.record.firstname;

This can be shortened a bit to

somename = database.customers.record.firstname;

The record structure contains a single record in the table. This structure needs to be populated before any record is written to the table (using idbOne, idbWrite, idbAdd, idbPut, idbMarkDelete or idbDelete).

When reading a single record (idbGet) then the record is populated with the result of the read. If the record is not located then the record is unaltered.

When reading a group of records (idbSelect), an a onRecord function is used, then the record structure is populated on each call to onRecord. If an onRecord function is not used, then the record structure is not primed, but all the records are instead sent to the resultset parameter.  For example;





Calls to idbSelect which do not pass an onRecord function will have the result sent to the oncomplete function. You can then loop through the result set, inspecting each record as you go.

You can iterate through the result set;

var row=0;
for (row in resultset){
    database.customers.record = resultset[row];
  // do something here
}


Two utility functions are also provided. These are mostly used for debugging purposes.

database.tablename.view()

This sends the contents of the table to the browser debugging console as a pipe separated list. This allows you to see the number of records in the table, and also the contents of the table. You can use this function directly in the browser developer tools area.

database.tablename.empty()

As the name suggests this empties the table in the browser. Again, this is mostly useful for debugging purposes. Bear in mind that if you are sync'ing data with a server then the data in the server will be re-fetched on the next sync. So if you empty the table locally either empty it on the server as well, or be prepared for the data to re-appear.

nt-idb.js

The reading and writing to the local data is done with a set of NetTalk JavaScript functions. These functions are located in scripts\nt-idb.js. If you wish to access the data directly from your own JavaScript code then you will want to make use of these functions, so they are documented here.


[End of this document]
Return to NetTalk Documentation Index