NetTalk FTP
Available in NetTalk Desktop, NetTalk Server and NetTalk Apps. Applies to Desktop
& Server (ABC or Clarion) apps.
Introduction
FTP is one of the oldest protocols on the internet. It is used to transfer files between and FTP Client and an FTP Server. NetTalk
includes an FTP Client Class which allows you to build FTP Client functionality into your Clarion program.
FTP is different to most other protocols because it involves two connections between the server and the client. The first connection
is known as the Control Connection and commands, and responses, flow on this connection. However when data needs to flow, a second
connection, known as the Data Connection is created. The actual file then travels on the Data connection.
Originally the data connection was opened by the server connecting to the client. While this worked well at the beginnings
of the internet, it requires special network configuration to allow this to happen. As the internet matured this reverse-connection
because a security hazard and most companies don't support it.
In order to extend the life of the FTP protocol a new approach, allowing the client to create the data connection to the server,
was invented. This approach is known as Passive mode. In builds before 8.29 the default for the object was Active mode. From build 8.29
the default is passive mode.
As security became a higher and higher priority the need to make the FTP
connection secure became important. The protocol was thus further extended
to add support for SSL (FTPS and FTPES) and support for SSH (SFTP). NetTalk
supports FTP over SSL (FTPS and FTPES) but does not support FTP over SSH.
NetTalk provides two classes which are used in the FTP support. The
NetFtpClientControl class handles the control connection, and the
NetFtpClientData class handles the data connection.
The FTP client
is always a Window procedure, because NetTalk communications are
asynchronous and thus need an ACCEPT loop, which in turn needs a window
structure. If the procedure is completely automatic then the window itself
can be hidden from the user.
Jump Start
This section gets you going as quickly as possible to add FTP functionality to your application.
- Make sure the Activate NetTalk Global extension has been added to the
application.
- Make sure the Activate StringTheory Global Extension has been added to
the application.
- Check out the ftpDemo app in the \examples\nettalk\ftp\FTP
Functions folder. This contains some example procedures which you may
be able to use in your application either "as is" or with modifications. See
the Examples section below for more on this example.
- If you want to construct an FTP procedure of your own, then the
following basic steps are pretty much required;
- Create a procedure based on the Window template
- Add a NetTalk Object Extension to the procedure, Object Name
ThisFtpControl and Base Class
NetFTPClientControl. On the Settings tab set
the Data Object to ThisFTPData
- Add a NetTalk Object Extension to the procedure, Object Name
ThisFtpData and Base Class
NetFTPDataControl. On the Settings tab set
the Control Object to ThisFTPControl
- Either set the Passive mode setting on the ThisFTPControl Settings
tab, or, in embed code, set the
thisFtpControl.PassiveMode property before the call to
ThisFTPControl.init. If this step is omitted
then the connection will default to Passive mode.
- Add some trigger event to the window (usually a button). Add code to
this button which primes all the required parameters (see Before Connecting below) and then calls
the first command you wish to execute (see Commands
below.)
- IIn the ThisFTPControl.Done method, add
code to execute when a command completes. (See Done below.)
- IIn the ThisFTPControl.ErrorTrap method
add code to deal with errors - often culminating in a
Post(Event:CloseWindow).
Before Init
The fundamental way the object works changes when it is in passive mode or active mode. Thus it is necessary to set the
PassiveMode property before the control object .INIT method is called.
The other properties can be set later, before the first command, but the
PassiveMode property must be set very early. The default value is 1, so if
this property is not set, or is set too late, then the FTP object will
assume that passive mode is required.
Before Connecting
As with most connections you connect to a server by specifying a host name and port number. If the server is not in passive
mode you should also set that and if the server is to connect over SSL then that needs to be set as well.
The server address is placed in the ServerHost
property. The server
port number is placed in the ControlPort property.
If the server supports
passive mode then the PassiveMode property is set to
True. This is the
default, so you don't have to set it. If the server does not support passive
mode then you need to set this property to False.
If you wish to
connect over SSL then call the SetFtpType method with one of the possible equates;
Ftp:NoSSL
Ftp:ImplicitSSLControl
Ftp:ImplicitSSL
Ftp:ExplicitSSL
The SetFTPType method takes the Data connection object as the second
parameter.
FTP is an authenticated protocol, meaning that a user name and password are required before you can start
transferring files.
Some servers allow the use of a "generic" login, called an anonymous login, where you can use some documented login and password,
however it's important to note that even these sites do require a login and password to be passed to the server.
The user login is placed in the property called
User, and the password is placed in the
property called Password. If these change
during a connection then set the property
OpenNewControlConnection to force the current control connection to
close and a new one to be opened with the next command.
If you are
connecting with SSL then all the normal SSL properties apply to both the
control connection and the data connection.
Example
This is an example of setting all the above settings before the Control
object is Initialized. In the code below thisFtpControl is the control
object, and thisFtpData is the data object.
thisFtpControl.serverHost
= 'ftp.capesoft.com'
thisFtpControl.ControlPort = 21
thisFtpControl.PassiveMode = true
thisFtpControl.SetFtpType(FTP:ImplicitSSL,thisFtpData)
thisFtpControl.User = 'someone'
thisFtpControl.Password = '1234'
thisFtpControl.openNewControlConnection = true
FtpControl.SSLCertificateOptions.CertificateFile =
''
FtpControl.SSLCertificateOptions.PrivateKeyFile = ''
FtpControl.SSLCertificateOptions.DontVerifyRemoteCertificateCommonName =
true
FtpControl.SSLCertificateOptions.DontVerifyRemoteCertificateWithCARoot =
true
FtpControl.SSLCertificateOptions.CARootFile = 'CA_Roots.pem'
FtpData.SSLCertificateOptions.CertificateFile = ''
FtpData.SSLCertificateOptions.PrivateKeyFile = ''
FtpData.SSLCertificateOptions.DontVerifyRemoteCertificateCommonName = true
FtpData.SSLCertificateOptions.DontVerifyRemoteCertificateWithCARoot = true
FtpData.SSLCertificateOptions.CARootFile = 'CA_Roots.pem'
Connecting
There is no need to explicitly connect to the server. The object will automatically connect as required when
a command is executed.
FTP over SSL
[ This section of the document has been updated to
be used with version 8.29 or later ]
SSL over FTP is done in one of two ways, Explicit and Implicit.
Implicit SSL (sometimes referred to as FTPS) is the simpler of the
two, each connection to the FTP server is done over an SSL connection. The
client program (ie NetTalk) assumes the server is listening using SSL, and
simply applies SSL to the connection from the very beginning. Because this
approach is simpler, it is also faster. Since most FTP-SSL servers that do
Explicit SSL also do Implicit SSL it's usually best to try this approach
first. Implicit SSL is included in builds 6.06 and later. To place the FTP
objects in SSL mode you should use the
SetFtpType
method with one of the following 4 equates;
Ftp:NoSSL
Ftp:ImplicitSSLControl
Ftp:ImplicitSSL
Ftp:ExplicitSSL
For example;
thisFtpControl.SetFtpType(Ftp:ImplicitSSL,thisFtpData)
Note that the second parameter is the Data Object.
These should be set After the call to
thisFtpControl.Init, but before any other FTP commands are called.
For Implicit SSL you will probably also need to set the PORT property to something other than
the default of 21. The default for Implicit SSL is port 990, however it may
be something else (and equally, servers running on Port 990 are not
necessarily SSL servers.) Explicit SSL servers may use port 21, or they may
use some other port.
Explicit SSL (informally referred to as
FTPES) is slightly different. In this mode the connection starts off as
normal (ie plain-text) but before the Login the client explicitly asks to
"go to" SSL mode. This asking is repeated at other times, such as when data
connections for uploading or downloading need to occur. Support for Explicit
SSL was included in NetTalk version 8.29 and later.
SFTP is something completely different,
it is the File transfer extension to SSH and is not supported at this time.
For more information on the flavors of FTP see
Wikipedia.
Control Template
From version 8.29 the control template has been deprecated. It still exists, and still works, but is not a
recommended approach
for use anymore. See the
JumpStart section for more information on adding FTP procedures to your application.
Commands
FFTP is designed to be a remote-file-system, meaning that the files are
organized with directories, and sub-directories and so on.
While you will primarily be sending and receiving files there are a number of commands you can use to navigate and manage the server.
The commands supported by the FTP object are listed here;
FTP Command | NetTalk Method | Comments |
abor | Aborting() | Sends an Abort command to the server.
This terminates any file being uploaded or downloaded. The
CommandQueue is emptied. No call to
Done or ErrorTrap
for the existing, or waiting commands will be called. A call to
Done with the _Command
property set to aborting will be made instead. |
appe | AppendToFile(LocalFileName, RemoteFileName) | Similar
to the PutFile, but does an append instead of an overwrite. It transfers the
local file specified by LocalFileName from the local machine to the FTP
Server, and stores it as RemoteFileName. If a file with the same path and
name as RemoteFileName already exists, the
transferred data is appended to the end of the existing file. If no such
file exists, a new file is created. Will either call Done or ErrorTrap or success or failure. |
cdup | ChangeDirUp() | Changes the remote current working
directory to one level higher in the directory tree. Will either call Done or ErrorTrap or success or failure. |
cwd | ChangeDir(DirectoryName) | Changes the remote current
working directory to DirectoryName. Will either call
Done or
ErrorTrap or success or failure. |
dele | DeleteFile(RemoteFileName) |
Deletes the remote file specified by
RemoteFileName. Will either call
Done or ErrorTrap or success or failure. |
list | GetDirListing(DirectoryName) | Retrieves a list of
contents of the remote directory specified by DirectoryName
and stores
them in self.DirListingQ property. To get contents of the current working
directory, pass an empty string as the parameter. For more information on
the DirListingQ property see Directory Listings
below.
By setting the
FullListing property to false, you will receive a list of file or directory
names only. When FullListing is true
(the default), full information on the files will be
retrieved.
Note that DirectoryName can also contain a remote file name.
In this case, current information on the specific file will be returned.
Note: Many FTP Servers are case sensitive
Examples:
GetDirListing ('') GetDirListing ('mydirectory')
GetDirListing ('myfile.exe') GetDirListing ('m*')
GetDirListing ('m*.exe')
Will either call
Done or ErrorTrap on success or failure.
|
size | GetSize(RemoteFileName) | Gets the size of a specific
file on the server. The result is placed in the
ExpectedFileSize property. Note that not all servers support this
call - if it fails ErrorTrap will not be called.
If it does fail then the code will check the current
DirListingQ to get a size from there. In all cases the
Done method is called once the command has
completed. |
mkd | MakeDir(DirectoryName) | Pass the name of the new
remote directory to be created as a parameter. Will either call
Done or ErrorTrap or success or failure. |
noop | Noop() | Not usually called by the program. Used
internally. NOOP is a non-operational command designed to do nothing. It is
useful for preventing a FTP Control Connection from closing. |
pwd | GetCurrentDir() | Returns the current directory name
on the server. The reply is placed in the CurrentDirectory
property. Will either call
Done or ErrorTrap or success or failure. |
quit | Quit() | This method will allow for all the data
transfer in process to complete and then disconnect you from the FTP Server.
Will either call
Done or ErrorTrap on
success or failure. |
retr | GetRemoteFile(RemoteFileName,LocalFileName) | Transfers the
remote file specified by RemoteFileName from the FTP Server to the local
machine and stores it in LocalFileName. If a file with the same path and
name as LocalFileName already exists, it will be overwritten by the received
file. A new file is created, otherwise.
Will either call Done or
ErrorTrap or success or failure. |
rnfr / rnto | Rename(RemoteName, NewName) | Renames the
remote file specified by RemoteName to
NewName. This method can also be
used to rename a directory. Will either call Done or
ErrorTrap or
success or failure. |
rmd | RemoveDir(directory) | Removes the remote directory
specified by DirectoryName. Will either call
Done or ErrorTrap on
success or failure. |
stop | Stop() | Immediately terminates the data connection. |
stor | PutFile(LocalFileName,RemoteFileName)
| Transfers the
local file specified by LocalFileName from the local machine to the FTP
Server, and stores it as RemoteFileName.
If a file with the same path and
name as RemoteFileName already exists, it will
be overwritten by the file being transferred. If the file does not already
exist, then it is created. Will either call
Done or ErrorTrap on
success or failure. |
| PutFile (StringTheory pBinData,RemoteFileName) | An
alternative version of PutFile. This one takes a StringTheory object as the
first parameter instead of a local file name. The contents of the object are
then stored on the server under the RemotefileName. |
syst | System | Gets a OS type from the server. the result
is placed in the SystemType property. |
type | SetType(Type) | The type is either A for ascii, or I
for binary. Not usually called by the program, is handled internally. Will
either call
Done or ErrorTrap on
success or failure. |
Properties
A number of properties affect how the FTP Client will behave. Other
properties are the result of commands.
Property | Description |
BinaryTransfer | If set to true then all file transfers will be in
binary mode. If false transfers will be in ASCII mode. The default is binary
mode, and there is almost never a requirement to transfer in ASCII mode. |
BytesLeftToReceive | When receiving a file, contains the amount of
the file which still has to arrive. |
BytesLeftToSend | When sending a file, contains the number of
bytes that have not been sent yet. |
_Command | Contains the command (as called in the command list)
which has just completed. Typically used in the Done or ErrorTrap methods.
Is also set at the start of the calls to PutFile or GetRemoteFile. Note that
this property is always in lower case. |
CurrentDirectory | Set after a call to the
GetCurrentDir command. |
DirListingQ | A queue containing the remote directory contents
after a call to GetDirListing. See
Directory Listings for more information. |
ExpectedFileSize | The size of the file being retrieved. Populated
by a call to GetSize, which in turn will use the
existing DirListingQ property if the server does not support the SIZE
command. |
ForwardSlashes | UNIX operating systems use the forward slash (/)
instead of the backslash (\) when separating directories and file names.
Since FTP originated on UNIX most FTP servers (even when running on windows)
follow this convention. If this property is true then all backslashes in
remote file name parameters will automatically be converted to forward
slashes before executing the command. |
PassiveMode | This needs to be set before the object
INIT method is called. (In other words, it's
called much earlier than any other setting). The default value is true. |
ProgressControl | The Use Equate of a progress bar control on the
window. Can be used to display the progress of file uploads and downloads.
See Progress Control for more information. |
SystemType | A text string containing the System Type as
identified by the server. A command to fetch this is called automatically
when logging into the server. See System. |
| |
Done
As motioned earlier the communications between the server and the client is
asynchronous. This means that you send a command, but you cannot immediately
test for the response to that command. Rather when the command completes
successfully the .Done method will be called. So if you wish to add more
commands at this point then you can do so.
Inside the done method you
can test the ._command property which will return the name of the command
which just completed. This property is in lower case, and matches the
NetTalk Method Name in the table above.
If the command is
unsuccessful then the .ErrorTrap method is called instead of the
.Done
method.
Example
case
self._command
of 'makedir'
self.changedir(somedir)
end
Important: Some commands that you
call will generate other commands internally. For example when doing a
Putfile a SetType
command is executed first. In this case you'll get a call to
Done first for SetType
and then later on for PutFile. So your
code in the Done method should be aware that
more calls to Done may be called than commands
you have executed.
ErrorTrap
The ErrorTrap method is called when an error occurs. when this happens the
Done method will not be called.
Directory Listings
The format of the directory listing is not specified by the protocol. So the
exact format of the text which is returned depends on the specific server,
and specific OS of the server. NetTalk does a reasonable job of deformating
this reply and placing the result in a property called .DirListingQ.
The Queue is declared as follows;
Net:FTPDirListingQType Queue, Type
Name
string(FILE:MAXFILENAME)
Shortname
string(13)
Date
long
Time
long
Size
long
Attrib
byte
NoOfLinks
long
Owner
string(50)
GroupOwnership
string(50)
AccessPermissions
string(50)
end
In your program you can declare your own queue of this
type and as long as it has the same structure, you can set the object to use
your queue rather than the built-in one. For example, in the program
code;
MyQueue
Queue
Name
string(FILE:MAXFILENAME)
Shortname
string(13)
Date
long
Time
long
Size
long
Attrib
byte
NoOfLinks
long
Owner
string(50)
GroupOwnership
string(50)
AccessPermissions
string(50)
end
thisFtpControl.DirListingQ &= MyQueue
This means that the call to GetDirListing will result in your queue
being automatically populated with the result. Since the queue is declared
in your procedure it is also easy to assign it to a LIST control on the
window.
Unknown format of directory listing
If the object is
unable to interpret the reply from the server then an error will be
generated "Unknown format of directory listing". It is quite unusual
to find a server that uses a format that NetTalk can't figure it out.
If you don't want this error to be called then put the following code in
the virtual method _FigureOutDirFormat() after
the parent call:
if self._DirListingFormat = 0
self._DirListingFormat = 255
end
You will need to figure
out the directory listing yourself in this case. The way that it works is
that the _ProcessGetDirListing() method is
called when data is received. If there is not enough information to figure
out the format then the _DirFormatSet property
is set to zero and the data is stored.
When there is sufficient
information from the server the _DirFormatSet
property is set to 1 and _FigureOutDirFormat()
is called. If this method manages to figure out the directory format then it
sets the _DirListingFormat property, otherwise
it sets it to zero.
This is what is happening in this case - enough
data has been received to figure out the format, but the format itself is
unknown. In order to add support for the custom format add code to the
_FigureOutDirFormat() embed to process the
directory listing after the parent call (the parent method could not figure
it out, which is what caused the error).
The _FigureOutDirFormat()
method is passed a string which contains the directory list (what you would
see printed out in a terminal if you were connecting to the FTP server on
the command line and the LIST command was issued to the server).
NetTalk allows you to process the listing and set the _DirListingFormat
to a number between 200 and 255, which means that it is in a custom format.
You then need to process this string in the _FillDirListingQ_FormatCustom()
virtual method that NetTalk provides for you.
It might be helpful to
have a look at the source code for the methods such as
_FillDirListingQ_Format01() to see how they process the string into
the queue and then write your own
FillDirListingQ_FormatCustom() to process your specific server's
response. You might find that it is quite close to one of the currently
supported formats. The full source code for the FTP objects is in the
NetFtp.clw and NetFtp.inc
files in your \Clarion\Accessory\Libsrc\win
folder.
Progress Control
Some of the operations, specifically
PutFile and GetRemoteFile
can take some time to complete. The class has a property called
.ProgressControl which can be set to the use
equate of a progress bar on the screen. If this property is set then the
control (ie a progress bar) will be updated as the file upload, or download,
happens.
For example
thisFtpControl.ProgressControl = ?Progress1
Three additional
properties are available if you wish to display the actual values. They are;
.ExpectedFileSize
.BytesLeftToSend
.BytesLeftToReceive
Examples
NetDemo.App
In \Examples\NetTalk\Demo folder.
This example
contains a window procedure called TestFTPClient. It is a highly visual
example of the FTP Client class which allows you to navigate an FTP server,
experiment with settings, and so on. It is an excellent way to get a feel
for FTP, and to test the behavior against a specific FTP server.
FTPDemo.App
In
\Examples\NetTalk\FTP\ABC folder.
This application contains examples of various "stand-alone" functions
that you might want to use in your application. These functions can be
imported into your own app for your own use. Current functions are;
function | use |
FtpFile | Write a file to an FTP server, or read a file from
an FTP server. Note that this function does one specific task - it's not
a good starting point if you need to perform a number of tasks against
the FTP server. |
FtpDirectory | Write all the files in a directory to the FTP
server, or read all the files from a directory on the FTP server. |
GetFileOverFTP | An example of calling the FtpFile function to
fetch a specific file from a specific server. |
GetDirectoryOverFTP | An example of calling the FtpDirectory
function to fetch a specific directory from a specific server. |
SendFileOverFTP | An example of calling the Ftpfile function
to write a specific file to a specific server. |
SendDirectoryOverFTP | An example of calling the FtpDirectory
function to write a specific directory to a specific server. |
| |
FTPLegacy.App
In \Examples\NetTalk\FTP\Legacy folder.
Same as FtpDemo.App above, but based on legacy template chain.
FAQ
1. How do I upload multiple files using FTP?
You can call
.PutFile multiple times. The commands will be
queued and executed one at a time.
[End of this document]