NetTalk FTP
Available
in NetTalk Desktop, NetTalk Server and NetTalk Apps. Applies to Desktop
& Server (ABC or Clarion) apps.
Introduction
FTP is one of the oldest protocols on the internet. It is used to transfer
files between and FTP Client and an FTP Server. NetTalk
includes an FTP Client Class which allows you to build FTP Client
functionality into your Clarion program.
FTP is different to most other protocols because it involves two
connections between the server and the client. The first connection
is known as the Control Connection and commands, and responses, flow on
this connection. However when data needs to flow, a second
connection, known as the Data Connection is created. The actual file then
travels on the Data connection.
Originally the data connection was opened by the server connecting to the
client. While this worked well at the beginnings
of the internet, it requires special network configuration to allow this
to happen. As the internet matured this reverse-connection
because a security hazard and most companies don't support it.
In order to extend the life of the FTP protocol a new approach, allowing
the client to create the data connection to the server,
was invented. This approach is known as Passive mode.
In builds before 8.29 the default for the object was Active mode. From
build 8.29 the default is passive mode.
As security became a higher and higher priority the need to make the FTP
connection secure became important. The FTP protocol was thus further
extended to add support for TLS (FTPS and
FTPES) and support for SSH (SFTP). NetTalk supports FTP over TLS (FTPS and
FTPES) but does not support FTP over SSH.
NetTalk provides two classes which are used in the FTP support. The NetFtpClientControl class handles the control
connection, and the NetFtpClientData class
handles the data connection.
The FTP client is always a Window procedure, because NetTalk
communications are asynchronous and thus need an ACCEPT loop, which in
turn needs a window structure. If the procedure is completely automatic
then the window itself can be hidden from the user.
Jump Start
This section gets you going as quickly as possible to add FTP
functionality to your application.
- Make sure the Activate NetTalk Global extension has been added to
the application.
- Make sure the Activate StringTheory Global Extension has been added
to the application.
- Check out the ftpDemo app in the \examples\nettalk\ftp\FTP
Functions folder. This contains some example procedures which
you may be able to use in your application either "as is" or with
modifications. See the Examples section below
for more on this example.
- If you want to construct an FTP procedure of your own, then the
following basic steps are pretty much required;
- Create a procedure based on the Window template
- Add a NetTalk Object Extension to the procedure, Object Name ThisFtpControl and Base Class NetFTPClientControl.
On the Settings tab set the Data Object to ThisFTPData
- Add a NetTalk Object Extension to the procedure, Object Name ThisFtpData and Base Class NetFTPDataControl.
On the Settings tab set the Control Object to ThisFTPControl
- Either set the Passive mode setting on the ThisFTPControl Settings
tab, or, in embed code, set the
thisFtpControl.PassiveMode property before the call to ThisFTPControl.init. If this step is omitted
then the connection will default to Passive mode.
- Add some trigger event to the window (usually a button). Add code
to this button which primes all the required parameters (see Before
Connecting below) and then calls the first command you wish to
execute (see Commands below.)
- IIn the ThisFTPControl.Done method,
add code to execute when a command completes. (See Done
below.)
- IIn the ThisFTPControl.ErrorTrap
method add code to deal with errors - often culminating in a Post(Event:CloseWindow).
Before Init
The fundamental way the object works changes when it is in passive mode or
active mode. Thus it is necessary to set the PassiveMode
property before the control object .INIT
method is called. The other properties can be set later, before the first
command, but the PassiveMode property must
be set very early. The default value is 1, so if this property is not set,
or is set too late, then the FTP object will assume that passive mode is
required.
Before Connecting
As with most connections you connect to a server by specifying a host name
and port number. If the server is not in passive
mode you should also set that and if the server is to connect over TLS
then that needs to be set as well.
The server address is placed in the ServerHost property.
The server port number is placed in the ControlPort
property.
If the server supports passive mode then the PassiveMode
property is set to True. This is
the default, so you don't have to set it. If the server does not support
passive mode then you need to set this property to False.
If you wish to connect over TLS then call the SetFtpType
method with one of the possible equates;
Ftp:NoTLS
Ftp:ImplicitTLSControl
Ftp:ImplicitTLS
Ftp:ExplicitTLS
The SetFTPType method takes the Data
connection object as the second parameter.
FTP is an authenticated protocol, meaning that a user name and password
are required before you can start transferring files.
Some servers allow the use of a "generic" login, called an anonymous
login, where you can use some documented login and password,
however it's important to note that even these sites do require a login
and password to be passed to the server.
The user login is placed in the property called User,
and the password is placed in the property called Password.
If these change during a connection then set the property
OpenNewControlConnection to force the current control connection
to close and a new one to be opened with the next command.
If you are connecting with TLS then all the normal TLS properties apply to
both the control connection and the data connection.
Example
This is an example of setting all the above settings before the Control
object is Initialized. In the code below thisFtpControl is the control
object, and thisFtpData is the data object.
thisFtpControl.serverHost = 'ftp.capesoft.com'
thisFtpControl.ControlPort = 21
thisFtpControl.PassiveMode = true
thisFtpControl.SetFtpType(FTP:ImplicitTLS,thisFtpData)
thisFtpControl.User = 'someone'
thisFtpControl.Password = '1234'
thisFtpControl.openNewControlConnection = true
FtpControl.TLSCertificateOptions.CertificateFile = ''
FtpControl.TLSCertificateOptions.PrivateKeyFile
= ''
FtpControl.TLSCertificateOptions.DontVerifyRemoteCertificateCommonName
= true
FtpControl.TLSCertificateOptions.DontVerifyRemoteCertificateWithCARoot
= true
FtpControl.TLSCertificateOptions.CARootFile
= 'CA_Roots.pem'
FtpData.TLSCertificateOptions.CertificateFile
= ''
FtpData.TLSCertificateOptions.PrivateKeyFile
= ''
FtpData.TLSCertificateOptions.DontVerifyRemoteCertificateCommonName
= true
FtpData.TLSCertificateOptions.DontVerifyRemoteCertificateWithCARoot
= true
FtpData.TLSCertificateOptions.CARootFile
= 'CA_Roots.pem'
Connecting
There is no need to explicitly connect to the server. The object will
automatically connect as required when
a command is executed.
FTP over TLS
[ This section of the
document has been updated to be used with version 8.29 or later ]
TLS over FTP is done in one of two ways, Explicit and Implicit.
Implicit TLS (sometimes referred to as FTPS) is the simpler of the
two, each connection to the FTP server is done over an TLS connection. The
client program (ie NetTalk) assumes the server is listening using TLS, and
simply applies TLS to the connection from the very beginning. Because this
approach is simpler, it is also faster. Since most FTP-TLS servers that do
Explicit TLS also do Implicit TLS it's usually best to try this approach
first. Implicit TLS is included in builds 6.06 and later. To place the FTP
objects in TLS mode you should use the
SetFtpType
method with one of the following 4 equates;
Ftp:NoTLS
Ftp:ImplicitTLSControl
Ftp:ImplicitTLS
Ftp:ExplicitTLS
For example;
thisFtpControl.SetFtpType(Ftp:ImplicitTLS,thisFtpData)
Note that the second parameter is the Data Object.
These should be set After the call to
thisFtpControl.Init, but before any other FTP commands are
called.
For Implicit TLS you will probably also need to set the PORT property to
something other than the default of 21. The default for Implicit TLS is
port 990, however it may be something else (and equally, servers running
on Port 990 are not necessarily TLS servers.) Explicit TLS servers may use
port 21, or they may use some other port.
ExplicitTLS (informally referred to as FTPES) is slightly different. In
this mode the connection starts off as normal (ie plain-text) but before
the Login the client explicitly asks to "go to" TLS mode. This asking is
repeated at other times, such as when data connections for uploading or
downloading need to occur. Support for Explicit TLS was included in
NetTalk version 8.29 and later.
SFTP is something completely different, it is the File transfer extension
to SSH and is not supported at this time.
For more information on the flavors of FTP see
Wikipedia.
Control Template
From version 8.29 the control template has been
deprecated. It still exists, and still works, but is not a recommended
approach
for use anymore. See the
JumpStart section for
more information on adding FTP procedures to your application.
Commands
FFTP is designed to be a remote-file-system, meaning that the files are
organized with directories, and sub-directories and so on.
While you will primarily be sending and receiving files there are a number
of commands you can use to navigate and manage the server.
The commands supported by the FTP object are listed here;
FTP Command |
NetTalk Method |
Comments |
abor |
Aborting() |
Sends an Abort command to the server. This terminates any file
being uploaded or downloaded. The CommandQueue
is emptied. No call to Done
or ErrorTrap for the existing, or
waiting commands will be called. A call to Done
with the _Command property set to
aborting will be made instead. |
appe |
AppendToFile(LocalFileName, RemoteFileName) |
Similar to the PutFile, but does an
append instead of an overwrite. It transfers the local file
specified by LocalFileName from the
local machine to the FTP Server, and stores it as RemoteFileName.
If a file with the same path and name as RemoteFileName
already exists, the transferred data is appended to the
end of the existing file. If no such file exists, a new file is
created. Will either call Done or ErrorTrap or success or failure. |
cdup |
ChangeDirUp() |
Changes the remote current working directory to one level higher
in the directory tree. Will either call Done
or ErrorTrap or success or failure. |
cwd |
ChangeDir(DirectoryName) |
Changes the remote current working directory to DirectoryName.
Will either call Done or ErrorTrap
or success or failure. |
dele |
DeleteFile(RemoteFileName) |
Deletes the remote file specified by RemoteFileName. Will either call Done
or ErrorTrap or success or failure. |
list |
GetDirListing(DirectoryName) |
Retrieves a list of contents of the remote directory specified
by DirectoryName and stores them in
self.DirListingQ property. To get
contents of the current working directory, pass an empty string as
the parameter. For more information on the DirListingQ property
see Directory Listings below.
By setting the FullListing property
to false, you will receive a list of file or directory names only.
When FullListing is true (the
default), full information on the files will be retrieved.
Note that DirectoryName can also
contain a remote file name. In this case, current information on
the specific file will be returned.
Note: Many FTP Servers are case sensitive
Examples:
GetDirListing ('')
GetDirListing ('mydirectory')
GetDirListing ('myfile.exe')
GetDirListing ('m*')
GetDirListing ('m*.exe')
Will either call Done or ErrorTrap
on success or failure. |
size |
GetSize(RemoteFileName) |
Gets the size of a specific file on the server. The result is
placed in the ExpectedFileSize
property. Note that not all servers support this call - if it
fails ErrorTrap will not be called.
If it does fail then the code will check the current
DirListingQ to get a size from there. In all cases the Done method is called once the command has
completed. |
mkd |
MakeDir(DirectoryName) |
Pass the name of the new remote directory to be created as a
parameter. Will either call Done or ErrorTrap or success or failure. |
noop |
Noop() |
Not usually called by the program. Used internally. NOOP is a
non-operational command designed to do nothing. It is useful for
preventing a FTP Control Connection from closing. |
pwd |
GetCurrentDir() |
Returns the current directory name on the server. The reply is
placed in the CurrentDirectory property.
Will either call Done or ErrorTrap
or success or failure. |
quit |
Quit() |
This method will allow for all the data transfer in process to
complete and then disconnect you from the FTP Server. Will either
call Done or ErrorTrap
on success or failure. |
retr |
GetRemoteFile(RemoteFileName,LocalFileName) |
Transfers the remote file specified by RemoteFileName
from the FTP Server to the local machine and stores it in
LocalFileName. If a file with
the same path and name as LocalFileName already
exists, it will be overwritten by the received file. A new
file is created, otherwise.
Will either call Done or ErrorTrap
or success or failure. |
rnfr / rnto |
Rename(RemoteName, NewName) |
Renames the remote file specified by RemoteName
to NewName. This
method can also be used to rename a directory. Will either call Done or ErrorTrap
or success or failure. |
rmd |
RemoveDir(directory) |
Removes the remote directory specified by DirectoryName.
Will either call Done or ErrorTrap
on success or failure. |
stop |
Stop() |
Immediately terminates the data connection. |
stor |
PutFile(LocalFileName,RemoteFileName) |
Transfers the local file specified by LocalFileName
from the local machine to the FTP Server, and stores it
as RemoteFileName.
If a file with the same path and name as RemoteFileName
already exists, it will be overwritten by the file being
transferred. If the file does not already exist, then it is
created. Will either call Done or ErrorTrap on success or failure. |
|
PutFile (StringTheory pBinData,RemoteFileName) |
An alternative version of PutFile. This one takes a StringTheory
object as the first parameter instead of a local file name. The
contents of the object are then stored on the server under the RemotefileName. |
syst |
System |
Gets a OS type from the server. the result is placed in the
SystemType property. |
type |
SetType(Type) |
The type is either A for ascii, or I for binary. Not usually
called by the program, is handled internally. Will either call Done or ErrorTrap
on success or failure. |
Properties
A number of properties affect how the FTP Client will
behave. Other properties are the result of commands.
Property |
Description |
BinaryTransfer |
If set to true then all file transfers will be in binary mode.
If false transfers will be in ASCII mode. The default is binary
mode, and there is almost never a requirement to transfer in ASCII
mode. |
BytesLeftToReceive |
When receiving a file, contains the amount of the file which
still has to arrive. |
BytesLeftToSend |
When sending a file, contains the number of bytes that have not
been sent yet. |
_Command |
Contains the command (as called in the command list) which has
just completed. Typically used in the Done or ErrorTrap methods.
Is also set at the start of the calls to PutFile or GetRemoteFile.
Note that this property is always in lower case. |
CurrentDirectory |
Set after a call to the GetCurrentDir
command. |
DirListingQ |
A queue containing the remote directory contents after a call to
GetDirListing. See Directory
Listings for more information. |
ExpectedFileSize |
The size of the file being retrieved. Populated by a call to GetSize, which in turn will use the existing
DirListingQ property if the server does not support the SIZE
command. |
ForwardSlashes |
UNIX operating systems use the forward slash (/) instead of the
backslash (\) when separating directories and file names. Since
FTP originated on UNIX most FTP servers (even when running on
windows) follow this convention. If this property is true then all
backslashes in remote file name parameters will automatically be
converted to forward slashes before executing the command. |
PassiveMode |
This needs to be set before the object INIT
method is called. (In other words, it's called much
earlier than any other setting). The default value is true. |
ProgressControl |
The Use Equate of a progress bar control on the window. Can be
used to display the progress of file uploads and downloads. See Progress Control for more information. |
SystemType |
A text string containing the System Type as identified by the
server. A command to fetch this is called automatically when
logging into the server. See System. |
|
|
Done
As motioned earlier the communications between the
server and the client is asynchronous. This means that you send a command,
but you cannot immediately test for the response to that command. Rather
when the command completes successfully the .Done
method will be called. So if you wish to add more commands at this point
then you can do so.
Inside the done method you can test the ._command
property which will return the name of the command which just completed.
This property is in lower case, and matches the NetTalk Method Name in the
table above.
If the command is unsuccessful then the .ErrorTrap
method is called instead of the .Done method.
Example
case self._command
of 'makedir'
self.changedir(somedir)
end
Important: Some commands that you call
will generate other commands internally. For example when doing a Putfile
a SetType command is executed
first. In this case you'll get a call to Done first
for SetType and then later on for PutFile.
So your code in the Done method should be
aware that more calls to Done may be called
than commands you have executed.
ErrorTrap
The ErrorTrap method is called when an error occurs.
when this happens the Done method will not be called.
Directory Listings
The format of the directory listing is not specified
by the protocol. So the exact format of the text which is returned depends
on the specific server, and specific OS of the server. NetTalk does a
reasonable job of deformating this reply and placing the result in a
property called .DirListingQ.
The Queue is declared as follows;
Net:FTPDirListingQType Queue, Type
Name
string(FILE:MAXFILENAME)
Shortname
string(13)
Date
long
Time
long
Size
long
Attrib
byte
NoOfLinks
long
Owner
string(50)
GroupOwnership
string(50)
AccessPermissions
string(50)
end
In your program you can declare your own queue of this type and as long as
it has the same structure, you can set the object to use your queue rather
than the built-in one. For example, in the program code;
MyQueue
Queue
Name
string(FILE:MAXFILENAME)
Shortname
string(13)
Date
long
Time
long
Size
long
Attrib
byte
NoOfLinks
long
Owner
string(50)
GroupOwnership
string(50)
AccessPermissions
string(50)
end
thisFtpControl.DirListingQ &= MyQueue
This means that the call to GetDirListing will result in your queue being
automatically populated with the result. Since the queue is declared in
your procedure it is also easy to assign it to a LIST control on the
window.
Unknown format of directory listing
If the object is unable to interpret the reply from the server then an
error will be generated "Unknown format of directory listing". It is
quite unusual to find a server that uses a format that NetTalk can't
figure it out.
If you don't want this error to be called then put the following code in
the virtual method _FigureOutDirFormat()
after the parent call:
if self._DirListingFormat = 0
self._DirListingFormat = 255
end
You will need to figure out the directory listing yourself in this case.
The way that it works is that the _ProcessGetDirListing()
method is called when data is received. If there is not enough information
to figure out the format then the _DirFormatSet
property is set to zero and the data is stored.
When there is sufficient information from the server the _DirFormatSet
property is set to 1 and _FigureOutDirFormat()
is called. If this method manages to figure out the directory format then
it sets the _DirListingFormat property,
otherwise it sets it to zero.
This is what is happening in this case - enough data has been received to
figure out the format, but the format itself is unknown. In order to add
support for the custom format add code to the _FigureOutDirFormat()
embed to process the directory listing after the parent call (the parent
method could not figure it out, which is what caused the error).
The _FigureOutDirFormat() method is passed a
string which contains the directory list (what you would see printed out
in a terminal if you were connecting to the FTP server on the command line
and the LIST command was issued to the server).
NetTalk allows you to process the listing and set the _DirListingFormat
to a number between 200 and 255, which means that it is in a custom
format. You then need to process this string in the _FillDirListingQ_FormatCustom()
virtual method that NetTalk provides for you.
It might be helpful to have a look at the source code for the methods such
as _FillDirListingQ_Format01() to see how
they process the string into the queue and then write your own
FillDirListingQ_FormatCustom() to process your specific server's
response. You might find that it is quite close to one of the currently
supported formats. The full source code for the FTP objects is in the NetFtp.clw and NetFtp.inc
files in your \Clarion\Accessory\Libsrc\win
folder.
Progress Control
Some of the operations, specifically PutFile
and GetRemoteFile can take some
time to complete. The class has a property called .ProgressControl
which can be set to the use equate of a progress bar on the screen. If
this property is set then the control (ie a progress bar) will be updated
as the file upload, or download, happens.
For example
thisFtpControl.ProgressControl = ?Progress1
Three additional properties are available if you wish to display the
actual values. They are;
.ExpectedFileSize
.BytesLeftToSend
.BytesLeftToReceive
Examples
NetDemo.App
In \Examples\NetTalk\Demo folder.
This example contains a window procedure called TestFTPClient. It is a
highly visual example of the FTP Client class which allows you to
navigate an FTP server, experiment with settings, and so on. It is an
excellent way to get a feel for FTP, and to test the behavior against a
specific FTP server.
FTPDemo.App
In
\Examples\NetTalk\FTP\ABC folder.
This application contains examples of various "stand-alone" functions
that you might want to use in your application. These functions can be
imported into your own app for your own use. Current functions are;
function |
use |
FtpFile |
Write a file to an FTP server, or read a file from an FTP
server. Note that this function does one specific task - it's
not a good starting point if you need to perform a number of
tasks against the FTP server. |
FtpDirectory |
Write all the files in a directory to the FTP server, or read
all the files from a directory on the FTP server. |
GetFileOverFTP |
An example of calling the FtpFile function to fetch a specific
file from a specific server. |
GetDirectoryOverFTP |
An example of calling the FtpDirectory function to fetch a
specific directory from a specific server. |
SendFileOverFTP |
An example of calling the Ftpfile function to write a specific
file to a specific server. |
SendDirectoryOverFTP |
An example of calling the FtpDirectory function to write a
specific directory to a specific server. |
|
|
FTPLegacy.App
In \Examples\NetTalk\FTP\Legacy folder.
Same as FtpDemo.App above, but based on legacy template chain.
FAQ
1. How do I upload multiple files
using FTP?
You can call .PutFile multiple times. The
commands will be queued and executed one at a time.
[End of this document]