SARAH- FTP client for ZX Spectrum

version: 1.1
author: Hood
release: 28.01.2024
download: version 1.1


******************************
TODO LIST ASAP (it means I will fix them as first before any other work on Sarah) 06.04.2024
  • TOP PRIORITY!!! make SARAH version for ZX Next
  • errors during work with directory (download is fine), quite annoying
  • issues with file name from BSDOS during upload (eg. "WM_BSD_inf")
  • issues with file name in esxdos during download (eg. *.AY, dots in file name, "Amaurote - v1.0.tap", etc)
  • issues with downloading files of big size (eg. 19256152 bytes)
  • issues with ftp servers which require precise ftp command length
    ******************************






    WHAT's NEW IN 1.1?
    Since eLeMeNt ZX was fitted with its own Wifi module option, and MB03+ has its own Wifi module also, there emerged an urgent need for the Wifi software to distinguish clearly between those two and to work flawleslly in whatever Wifi presence in both devices. So if you have Wifi module in eLeMeNt ZX only, or in MB03+ only or even both Wifis are present in eLeMeNt ZX and MB03+, the Wifi software should work perfectly now.

    - new: Sarah 1.1 now exists for eLeMeNt ZX version, works without compromise. IMPORTANT!! Set Expansion pack B in your eLeMeNt ZX setup for "UART MB03+", otherwise Sarah 1.1 will not work! (see picture above).
    - change: new Wifi bios 1.9 inserted. It enables to easily switch between all versions of Sarah no matter what Wifi module presence you have
    - bugfix: speed managemenet call earlier (caused endless loop if wifi was off)
    - added: Sarah 1.1. client compiles to 4 version now.
  • MB03+ BSDOS (MBBSD) for Wifi module in MB03+, use it with BSDOS
  • MB03+ ESXDOS (MBESX) for Wifi module in MB03+, use it with Esxdos
  • EL+ BSDOS (ELBSD) for Wifi module in eLeMeNt ZX, use it with BSDOS
  • EL+ ESXDOS (ELESX) for Wifi module in eLeMeNt ZX, use it with Esxdos
    Abbreviations MBBSD, MBESX, ELBSD and ELESX appear in Sarah's title line to easier distinguish between them.




    version: 1.0
    author: Hood
    release: 06.07.2023
    download: version 1.0
    video: SARAH acessing local ftp server
    SARAH acessing remote ftp server
    SARAH acessing C64 Ultimate IIL+
    SARAH acessing ftp server running on a mobile phone
    SARAH acessing SARAH server (a brand new server specially designed for usage on ZX Spectrum directly)






    FOREWORD
    Dear Wifi users, with big happinnes and proudness in my heart, I present to the ZX world one of the first FTP clients for ZX Spectrum, called SARAH. It took me 18 months of ceaseless work so far. The result is quite satisfying. I regard it is probably the most useful program I have written for ZX Spectrum so far.

    Previous public version 0.7 was not able to display and browse directory content. Version 1.0 can do it now.

    I designed SARAH 1.0 in a way, so that it can be ported to another hw configuration most easily. In version 1.0, I may only fix bugs or add something minor but do not plan to extend it anymore. Instead, I plan next versions specially bound to eLeMeNt ZX and MB03+ Ultimate (www.128land.com) hardware to make the SARAH better. So if you are interested in porting version 1.0 to another ZX system, version 1.0 is practically the best (and only) starting point. In that case, just ask me for details.

    SARAH 1.0 is able to run on ZXS 48K. It has only tens of bytes free space, but I want to keep it for bug fixing.

    The main code of SARAH 1.0 uses latest Wifi Bios ver 1.8. and Speed Management 2.0. The Bios ensures low level routines and the Bios as well as Speed Management 2.0 can be used independently in other applications too. Both make it easier for wifi coder to code their programs.

    The source code consists of extra sources: binary font, keyboard and print source, keyboard read routine by Busy and disk opeartion sources for easier porting to another disk systems.

    If porting, keep in mind, that the wifi device we use is placed in MB03+, it is ESP-01 module, the same one as was used in first batch of ZX Next. Writing your own wifi bios for your wifi hw would be the easiest option to port SARAH to hw of your favour. Recently, also eLeMeNt ZX has been fitted with a bit more modern wifi module, so SARAH should be able to work also without MB03+ interface. But I have not prepared it yet, hopefully will be available very soon in future release.

    SARAH 1.0 exists in two variants. They differ in disk system support. The first is for BS-DOS, the second for Esxdos. So I guess the later will be mostly used as Esxdos users are many. About disk operations later on.



    REQUIREMENTS
    - supported hardware: Wifi module ESP-01 in MB03+ or eLeMeNt ZX
    - supported OS: BSDOS, esxDOS
    - memory: standart 48K RAM



    ACCESS TO SARAH SERVER
    Like most of FTP regular servers, you need to login to SARAH with user and password. Simply download SARAH client and access info has been pre-installed in it. So if you run SARAH client, it should login and connect you with SARAH server automatically. USER and PASSWORD are then visible on SARAH client screen.



    MAIN FEATURES
    SARAH is ideal for downloading and uploading ZX programms between FTP server and your ZX computer, without need of removing your memory card. It also enables you to organise directories and files on the FTP server.

    A special server called SARAH has been prepared for us with over 8.500 ZX titles. Its address is: sarah.speccy.cz. It is public and run without profit. As far as I know, it is first of its kind ,it contains only files that ZX can run, ie. mainly TAP, TZX, SCR, POK, TXT and others. More info on here.



    FEATURES
    - it can connect to local, remote as well as mobile ftp servers
    - it uses 20 ftp commands (previous release 0.7 had 10)
    - it uses command prompt for you to write other commands directly to the server
    - display and browse directory
    - mark files
    - find files
    - upload one file
    - download multiple files
    - for download it uses passive transfer mode only
    - create directory
    - rename files/ directories
    - delete files/ empty directories
    - maximum items that the directory is able to hold up is roughly 500- 1200 items






    ZX MEMORY
    SARAH 1.0 executive code starts from #C800 (51200). It also uses buffer memory from #6000 to #C7FF which is used for upload and download files and for directory. Since this memory is shared, directory has to be reread each time after file has been up/downloaded. Executive memory could be extended, but only at cost of diminshing buffer memory, which would result in less place for directory items. Stack pointer starts at #6000 and goes down. For multiple item operations, memory from #4800- #4FFF is in use too.



    CONTROL KEYS
    Control keys can be divided into 2 groups. First group is controls 1) without directory read, second group of keys 2) controls for directory operation.

    1)- general control keys
    - F- file: allows user to enter their default file name which will be downloaded always when pressing SS+D
    - T- start: allows user to enter their default start address for files being downloaded (not applicable to Esxdos)
    - A- autologin: select if automatic login should be carried out upon SARAH's start
    - H- host: select your ftp server URL
    - E- usr: enter ftp server user
    - P- pass: enter ftp server password
    - SS+P: show/hide displayed password
    - U- upl: upload file, enter its name
    - S- size: returns item size, enter its name
    - Q- quit: quit SARAH (ftp connection remains up)
    - W- wifi: on/off. Sets your wifi state upon SARAH's quit
    - L- login login to ftp server, user and password will be entered manually
    - SS+L login to ftp server, user and password are taken from variables
    - O- logoff logg of ftp server
    - V- savApp save application with its settings and variables
    - D- dwnld download file/s
    - ss+D download file, name and start address taken from variables
    - CS+D download file, name entered manually
    - C- com enter ftp command
    - ss/CS+M un/mark all


    2)- directory control keys
    - I- Fn find item
    - Spc space- un/mark item
    - Z,X roll directory item if it is longer than screen
    - R- Rr reread directory
    - 6- Rn rename item
    - 7- MD make new directory
    - Del delete item/s
    - Br- UD break key, jump up directory
    - N change way of directory retrieve (NLST/ LIST)
    - ss+N read directory abbreviated or full
    - Enter enter directory, or display file 6912 or 6144 bytes long
    - arrows up/down for browse directory up/down or left/ right for page up/down


    In the course of your work, where available, use B key to abort action.



    SCREEN LAYOUT
    The screen can be divided into 3 parts. Upper, middle and lower.

    Upper part of the screen:
    - line 1,2 Author+ version number, or directory path.
    - line 3 Speed: speed info (default is 16, which is max), speed cannot be changed
    Logged in: OK for being logged in, X the opposite
    STATUS: 2- module connected to AP and IP is obtained
    3- TCP transmission created (this is the state you want)
    4- TC transmission disconnected
    5- module is not connected to AP (Acess Point) at all
    - line 4 Host: shows URL of your ftp server, or X if not connected
    - line 5,6,7 answers from FTP server


    Middle part of the screen:
    - line 8-15 FTP directory


    Lower part of the screen
    - line 16 related to FTP directory.
    showing number of items/ directories/ files
    directory control keys
    - line 17: File: variable file name, that the user can change.
    Start: variable start address for downloaded file, that the user can change
    - line 18: Host: variable host name, that the user can change
    User: user name for login, that the user can change
    Pass: password for login, that the user can change
    - line 19,20: general control keys
    - line 21: number of marked files
    directory retrieve mode (LIST/NLST/full/abbr)
    - line 23: info line guiding the user throughout their action
    - line 24: input line


    INPUT LINE FEATURES
    Throughout your work input prompt line apperas on the screen. The input line uses these keys:
    - left/right- for moving your cursor
    - TRUE VIDEO- for input abort
    - DELETE
    - GRAPH- for backspace
    - SS+Q- for jump on line start



    SETUP
    There is no setup or configuration file. SARAH 1.0 works as a single machine code file (on MB03+) or a .TAP file (on Esxdos). The reason is, that on BS-DOS you will be able to run it from a remote path (using eg. NEW#3,25 command, meaning "run file number 25 from disk 3). More info on how to handle setup, see below in disk operations section.






    LOGGING IN
    SARAH 1.0 has quite convenient way of logging in to ftp server. First of all, SARAH 1.0 contains host url, user and password variables, which the user can change. At the same time, you can set auto login variable with A key. If set to yes, SARAH 1.0 performs automatic login to ftp server. If set to no, it does not.



    SERVERS
    Communicating with server is probably the trickiest part of SARAH 1.0. Different server types and settings or restrictions can make it impossible or hard to access such servers. SARAH 1.0 should be able to access any ftp server in the world, including mobile phone ftp servers, which do not operate on port 21 (standart ftp port), but on a different ports. ON ZX Spectrum, we are quite dependent on the server as 8 bit ZXS does not have enough power to organise obtained data. Theoretically, any ftp server can behave in a different way. If you run upon the server, of which data SARAH 1.0 processes badly, let me know.

    In case your server is a standart one (ie. it uses standart FTP port 21), just enter it with key H in normal way. Like "192.168.0.200", or "hood.speccy.cz". If your server uses different port (eg. 31546), enter its URL like this: "192.168.0.200:31546", or "hood.speccy.cz:31546".



    DIRECTORY
    I think, work with directory should be the essence of any FTP client. Although we are quite limited with ZXS possibilities how to process incoming directory data. Out of the few servers I tried, all of them returned directory items (be it directory or file) in alphabetical order, which is a nice feature for the user. I found no way, how SARAH 1.0 would make the server send data in a way SARAH 1.0 dictates. Almost all servers I tested did not distinguish between directory and file. All were just items. It means, directories starting with eg. Z letter were sent at the end of directory list. The only one exception were mobile phone servers, some of them returned directories first, and then files- all in alphabetical order, which, I believe, is an ideal. Organising items in different way would require very sophisticated routines on ZX side, more memory and long waiting time. I think, we can forget about it for the moment.



    WORKING WITH DIRECTORY
    Directory is closesly bound with memory. The free memory for directory is #6800 bytes (26624 in decimal). It often happens this is not enough even for ZX files to hold them all. If SARAH 1.0 runs out of memory during directory retrieve, it will show a message "Transfer incomplete". Then there is an information about number of directories or files or items stored in your memory displayed just a line below directory itself.

    Retrieving directory can be done in 3 modes (see ss/+N control):
    1) LIST full
    LIST is an ftp command that returns complete items (like: attributes, owner, date, time, length, etc.). This mode brings you all the data. Use this mode only in case other modes fail, it shows you the way the server returns the directory. In this mode SARAH 1.0 cannot work properly, since the data can varry a great deal. Some example could look like this:

    -rw-r--r-- 1 ftp ftp 27582 Feb 03 2022 03.lim
    -rw-r--r-- 1 ftp ftp 30913 Feb 03 2022 03.scr
    drw-r--r-- 1 ftp ftp 22445 Feb 03 2022 04
    -rw-r--r-- 1 ftp ftp 30913 Feb 03 2022 04.scr

    2) LIST abbr (the default mode)
    same as mode 1, but redundant data will be removed this time, meaning ZX memory can hold more items. Directories start always with "D" at the line start. This is the default mode and could look like this:

    27582 Feb 03 2022 03.lim
    30913 Feb 03 2022 03.scr
    D 22445 Feb 03 2022 04
    30913 Feb 03 2022 04.scr

    3) NLST full
    NLST is an ftp command and it returns item names only. For us this is the most economical way of storing items. This mode is always reliable, but you cannot differ between a directory and a file, that's why I reffer to them as items. However, entering an item that may look like dir, SARAH 1.0 will try and if it is directory, it should not fail. The output looks like this:

    03.lim
    03.scr
    04
    04.scr

    It can happen, one directory entry is longer than the whole screen. You recognise such an entry by two ending dots. In this case, press "X" and "Z" to roll through the item to see it complete.



    DELETING DIRECTORY
    If the directory is empty, there is no issue deleting it. If not empty, FTP refuses to do it. The directory have to be entered, all items marked and deleted. However, in SARAH 1.0 you have to do it manually. There is a lack of memory for such action.



    JUMPING FROM DIRECTORY
    You can either move the cursor to the very first item in the directory ".." and press enter, or more convenient, press break key at any time, it does the same.



    MARKING ITEMS
    With the space key, you can mark or unmark files. Pressing SS/CS+M marks/unmarks all files. Other ways of marking files is with find function, see below.



    FIND
    This is very useful function, lifting SARAH a level higher at only several bytes cost. You may enter any kind of string, SARAH will look up the item name for you in the directory. Matching items are highlithed in red and their number will be displayed on the third bottom line. At the same time you will be transfered to the first matching item. Find function is case insensitive. If you enter just one letter, all items starting with this letter will be found- this is very useful, if you just need to move to certain letter in the directory file list. Found items will become marked items.

    Let's say you want to find all tap files. Simply enter ".tap" and all matching files will be highlighted for you.



    UPLOAD
    In SARAH 1.0 as opposed to the previous 0.7 I added STOU command upon which the server creates a new file in case the name already exists. Press U, enter file name from your disk and upload will start. Speed is around 3 kb/ sec.



    DOWNLOAD
    There are 3 was of how to download file.

    SS+D= download file from file name variable and assign it with start address from the variable. This is very handy when you eg. need to test one file (eg. you are writing an executive code on PC) and want to download it all time around.

    CS+D= enter file name you want to download manually.

    D= download file under cursor. If there are marked files, than the one under your cursor will be ignored and all marked files will be downloaded.

    Download speed using 20 Mhz CPU should be up to 60 kb/sec (depending on ftp the server).



    DISK OPERATIONS
    You will find two disk versions of SARAH 1.0 in the package. The first one is for BS-DOS, and the second one for Esxdos. Title line has BSD or ESX at its end, to distinguish between them.
    The first version runs on MB03+ interface with BS-DOS. BS-DOS originated in 1996 and was written by Slavo Labsky, known as Busy, introduced in MB-02+ hardware, the predecessor of MB03+. This is still my main disk system, I am using it in 99% when sitting at Speccy. SARAH 1.0 in BS-DOS version is just one machine code file which can be run with NEW command. It means if you want to upload or download files from/ to your favourite destination, just navigate yourself to that destination from Basic and run SARAH 1.0 with usual "NEW#disk_number,SARAH_file_number" and your path remains unchanged. In order to navigate through your disks and directories more happilly, use my nmi menu, which enables to change disks and directories with keys O and P. If the file being downloaded already exists on the disc, this presents no obstacle for BS-DOS. It can hold hundreds of files carrying the same name.

    The setup file is saved as a single machine code file as well.

    Similarly, in Esxdos, you will have to navigate through your source/destination path using nmi menu, quite easily.

    Uploading files from your Speccy to FTP server is the same for both systems. Press U to enter the name of the file in your current disc location which you wish to upload, and it will be read from your disc and uploaded to the ftp server to the path you have opened.

    In download, files travel the oposite way. While on BS-DOS, same name files are no issue, on Esxdos, if the file being downloaded already exists on your destination disc, its name will be automatically rewritten in its last 4 positions. Eg. file "manicmin.tap" will be renamed "mani#001.tap", then "mani#002.tap", etc. You can then rename the files manually in nmi menu, or LNX commander, or in Basic.



    DIRECT COMMAND INPUT
    True, that SARAH 1.0 uses 20 ftp commands. But there are others, that I have not implemented. I avoided their implemenattion in order to save memory. Instead, you can type them using C controll key, case insensitive. Thus, you may enter eg. command SITE, upon which the server returns you all FTP commands supported by the server. Commands which need 2 channel work (communication and data channel, like LIST, RETR...), will not work in direct command input. They work in the main program instead.






    LOCAL FTP SERVER
    It is advisible to install your local server to be able to copy files from your ZX to PC and vice versa. In the package you will find FileZilla server and its interface which I find both, small and easy to install and set.

    In the server interface, you need to add a new user and password, assign a path to your drive on PC that the client will work with and enable work with files (delete, append, etc.) see the picture below. I found the latest versions incovenient to set, so I use this old version instead. But practically, you can use any local ftp server of your wish.





    IDDLE TIME
    SARAH 1.0 sends a NOOP command roughly every 1 minute to keep your connection alive and to prevent being kicked off by the server for inactivity. However, some servers can monitor traffic and kick you off, despite NOOP command sending.



    ERROR CODES
    Serve mainly for my purposes, but you may encounter them, here is the list for you:

    ;1=sending string error (cipsend_request routine) ...OK
    ;2=error sending CWMODE AT commad ...OK
    ;3=error sending CIPMUX AT command ...OK
    ;7=error sending CIPCLOSE AT command (ftp upload) ...OK
    ;8= module did not return SEND OK after data block sending (ftp_upl_send)..OK
    ;9= error in entering passive mode (enter_passive_mode) ...OK
    ;14=file not exist ...OK
    ;15=disk error ...OK
    ;25= module not connected to AP, the user has to set his module externally.OK
    ;ftp ;4= check_module_answer routine did not found the answer from DE ...OK
    ;5= uploaded data not sent correctly (ftp_upl_send) ...OK
    ;6= cipsend_lod routine error. Module not ready to send data for upload ...OK
    ;10= error in connecting to ftp server (ftp_login)(probably wrong URL) ...OK
    ;11= error in receiving ftp answer (receive_ftp_answer) ...OK
    ;12= "+IPD," string not found in the incoming bytes (look_for_ipd) ...OK
    ;13= FTP server did not confirm answer 225 on download attempt (ftp_download_file, or retrieve directory...OK
    ;16= ABOR command not successful ...OK
    ;17= upload: connection not accepted (code 150 not received) ...OK
    ;18= upload: transfer not ok (code 226 not accepted) ...OK
    ;19= initial check. Connected to server, not good answer to NOOP cmnd ...OK
    ;20= ABOR command initiated by the user ...OK
    ;21= download: files does not exist ...OK
    ;22= file does not exist on ftp server ...OK
    ;23= download: block of data receive error ...OK
    ;24= server did not respond with "226 Transfer OK" ...OK
    ;25= command sent ok, but return code other than expected ...OK
    ;26= no data available in passive receive mode (routine 84) ...OK
    ;27= command for create dir not accepted (create dir)
    ;28=speed management: AT command not accepted ... present in the code but not used
    ;29=speed management: all speeds tested, timeout, fix it in terminal-add it in the next ver... present in the code but not used
    ;30=speed management: failed, probably Wifi module wrongly set,fix in terminal-add it in the next ver... present in the code but not used
    ;31=user defined speed_ AT+UART_CUR command not sent ...???
    ;32=user defined routine_ speed in UART not set ...???
    ;33=user defined speed_ you try to set nonsence speed (ie. > 23) ...???
    ;34= dir retrieve error
    ;35= (look_for_ipd) 2nd error
    ;36= error in receive_ftp_answer
    ;37= send_ftp_command_in_passive_mode
    ;38= error sending TYPE command
    ;39= error in sending AT+CIPRECVMODE=0(1), (passive_receive_switch) ...OK
    ;40= error in sending command in ftp_quit_connectio ...OK
    ;41= error in sending command in enter_passive_mode ...OK
    ;42= error in sending command in send_noop ...OK
    ;43= channel to sclosed sucessfully ...OK



    BUGS
    There are bugs in SARAH 1.0. I spent months trying to kill them all. But some bugs still remained. If you encounter them during directory reading, press R to reread directory. Another bug occurs in Esxdos version. Some ftp filenames cause problems. Try to download file "Amaurote - v1.0.tap", Esxdos returns error code 15. I will work on this in the very next release. If you get stuck otherwise, press reset and start again. Sarah's bugs do not make any harm, they are just like any other bugs, make your life a bit harder. I continuusly work to kill'em all:) But since they occur randomly it is hard to simulate them.



    ACKNOWLEDGEMENTS
    Primarily:
    Busy for his couple of handful routines and advices, his idea of directory items conception, advices about find function.
    Lanex for his useful hints and advices on how to write Esxdos disc routines.
    LMN- for inspiration and knowledge.
    SCJoe- for inspiration in client's name choice
    Wixet- for dedicating me his time and space on his speccy.cz domain






    SOME USAGE DESCRIPTION/ FAQ/ TROUBLESHOOTING/ TIPS
    SARAH is self contained as far as basic wifi connection and setting is concerned. Upon its start, SARAH will automatically login to the prefered ftp server, if selection A is switched to yes. If not, it will let you login manually. You do not have to do any wifi module setting prior to running SARAH. The only thing that is required for automatic login is, that your module should have enabled auto AP, ie. should automatically connect to AP (access point). If it cannot connect automatically to an AP, you will have to do it either manually in the terminal or you can use my utility Wifi Setup.. From that time, no other action concerning wifi module settings will be required from you.

    Status 5. If your status remains 5, than your module is not connected to an AP, so most probably it has not set automatic connection to an AP. You will have to solve it outside ZX, in the terminal     terminal or Wifi Setup..

    A tip for quick download of a file: I use this feature especially when I am coding a project on a PC and the file to be downloaded has the same name (and start address). Set your client to autologin. And after it has logged in, press ss+D to download automatically your file from File and Start variable. Very easy, very convenient to work with.

    Setup: use V to save your whole client with its setup. It will be saved to your disk as a new file. In the setup you may change: file name, start address, host, user, pasword, autologin to ftp server upon start or during your work, wifi module on/ off on the client's quit, hide/disclose password. Simply, the state your saved your client will reapear on your next client's run.

    Autologin: how does it work? The client on its start will carry out automatic login to the host if autologin is set to yes. It can be also used during work with the client. Press ss+L and autologin will be made automatically for you.

    A tip: if you want to use the client permanently during your current work on ZX, switch your wifi module to ON with a key W. Then you can leave the client with QUIT only. This means you remained logged in to ftp server. Then you can do your other work on ZX. If you set your ftp server to no kick off, your ftp session will remain untouched, and you will enter it upon next run of the client.



    DATA FLOW SPEEDS
    Download: on a standart ZX machine with 3,5 Mhz CPU, SARAH manages speed of 16 kb/ sec including saving to disk. Yet I believe this could be increased a bit. For upload it manages mere 3,3 kb/ sec including load from disk. I doubt if it can be increased. The limit size our ESP-01 module can send in one go is 2048 bytes. So, paradoxically, ZX Spectrum is waiting for module here.

    However, SARAH changes CPU speed during download, if your ZX machine is     eLeMeNt ZX, so with 20 Mhz CPU speed the download speed including save to disk is 60 kb/sec. For upload, I did not manage to make it work with 20 Mhz, or even with 7 Mhz. Hopefully, this can be solved and improved in next versions.