20 Exchange of Data

IQdesktop is the working horse, providing the workflow tools, the computational power by the host system, and allowing the analyst to conduct the work to be done.

Storage of any data, models, or results should not happen on the virtual system and backup of files on the virtual system is not done. Transient storage is allowed. Several possibilities to efficiently and easily handle transient storage, long term storage, and backup exist:

  • Storage only outside the container via shared volumes (potentially slow when shares not on local computer)
  • Transient storage in IQdesktop with synchronization to the outside (recommended for server and cloud based deployment)

20.1 Storage on shared volumes

The configuration file for IQdesktop allows to define a folder that is shared with the host computer’s file system. By default only one such shared volume is set up but multiple are possible. The shared folder can be accessed from IQdesktop as \IQDESKTOP\SHARE.

This approach is recommended when using IQdesktop on your local computer.

It can also be used easily when deploying IQdesktop on a local server. In this case different file servers might be mounted to the the host computer and selected folders cane be shared with IQdesktop containers through the volume approach.

20.2 Mounting inside IQdesktop

File servers might be mounted directly from within an IQdesktop container. For this the privileged entry in the config file needs to be set to true.

20.2.1 AWS S3 Buckets

AWS S3 buckets can easily be mounted to IQdesktop. You will need to provide your AWS Access Key and your AWS Secret Access Key, either in the config file or by calling the command awskeys from the IQdesktop command line.

  • An AWS S3 bucket "s3bucketname" can then be mounted by typing mountS3 "s3bucketname".
  • It can be unmounted by typing umountS3 "s3bucketname".
  • In the commands above exchange "s3bucketname" with the bucket name that you have on AWS S3 and want to access.

After mounting, the mounted AWS S3 bucket filesystem will be available in: /IQDESKTOP/MOUNT/"s3bucketname".

20.2.2 Others

Mounting of other file systems is possible as well but not supported by implemented commands, as for AWS. You might need sudo rights on IQdesktop to accomplish this.

20.3 Git

When running IQdesktop on a server, IntiQuan recommends to use the shared folder (folder, volumes) to share bare git repositories that allow to clone them into IQdesktop. The proposed destination of the cloned git repositories is ~/PROJECTS, where ~ is the IQdesktop user’s home folder.

Ongoing work can be saved by commiting and pushing changes to the outside of IQdesktop. In addition to saving work outside IQdesktop, this approach also allows collaboration between different (IQdesktop) users on the same project.

Convenient git command aliases are provided in IQdesktop:

  • gclone equals: git clone
  • gpull equals: git pull
  • gpush equals: git add .; git commit -m "-"; git push
  • gpush message equals: git add .; git commit -m "message"; git push

20.4 Subversion

Since version 0.3.0 subversion is available on IQdesktop, allowing to checkout, commit, etc. files located on an external subversion server. Standard command line calls to the subversion client svn apply.

20.5 Secure copy

The secure copy protocol (scp) can be used to exchange data between an IQdesktop container and another computer. The SSH server in the IQDesktop container needs to be enabled for that.

On Windows the use of the WinSCP software (obtain from here) can be used for this purpose.

To connect to an IQdesktop container with WinSCP:

  • Enter the IPADDRESS of the containers host computer
  • Change the Port number to the SSH port number on your container
  • Enter your username and password
  • Press “Login”

You will be connected to your home folder on the IQdesktop container and can transfer files back and forth in a secure manner.

20.6 Map as network drive

Your IQdesktop home folder can be mapped as a folder in your local computers explorer. This allows for an additional possibility of file exchange that does not require the use of the \IQDESKTOP\SHARE folder.

  • Linux and Mac users will be able to mount their IQdesktop home folder via sshfs from the file manager. That approach allows to access IQdesktop as if it were a folder on the local machine giving you immediate access to all results produced on IQdesktop.

  • On Windows sshfs is not available by default but can be installed and used as described below.

20.6.1 Installation and use of sshfs on Windows

The installation procedure is documented at this location. Below abbreviated instructions are given that should be sufficient for a successfull installation.

Note that this installation does require administrator rights.

20.6.2 Mapping an IQdesktop container on Windows

  • Open the Windows Explorer. Right-click on This PC > select Map Network Drive and enter the address: \\sshfs\user@!6900

Note that should be used if you run IQdesktop on your local computer. If you want to access an IQdesktop instance that runs on a server you will need to provide that servers IP address or URL name. 6900 stands for your SSH port on your IQdektop instance. On local IQdesktop instances this is typically 6900 (but you can set this differently in the configuration file). On server installations your system administrator will provide you with the correct port number. Also exchange user with your username on your IQdesktop container.

You will be connected to your home folder on the IQdesktop container and can transfer files back and forth in a secure manner.