Disk Space and Performance

Performance

For best performance, fast SSDs are recommended. NVMe technology offers the best performance (N.B. NVME disk can typically read at more than 5000MB/s  which is 10x faster than a SSD, almost the speed of a decent RAM).
You should also configure these antivirus exclusions.

Disk Space

If Omniscope prompts a warning about "low disk space" or you notice that your system is running very low on storage space, there are different things you can do to fix or mitigate this in Omniscope.

1. Change Omniscope folders location to another disk

In case you have another disk available you can change Omniscope working / project folders location to be on other disks by editing settings in the Admin app.

N.B. you are responsible for deleting / moving the old existing files from the previous location.
        You will need to restart the app after editing these settings for them to take effect.

1.1  Sharing folder

The Omniscope sharing folder is the root containing all your projects, listed in the Omniscope projects page when you launch the app.

By default it's named "files" and it's located in your home directory under the "omniscope-server" folder. E.g. C:\Users\Freddie\omniscope-server\files

To customise the path navigate to the Admin app -> Web Server settings page ( /_admin_/wsgeneral/ )  and browse to the desired folder

1.2  Data Engine and Workflow app temp folders

Omniscope automatically manages the Data Engine and Workflow app temp space, performing periodically checks to clean up stale / least used resources when the disk is low in space.

Still a disk running out of space could cause app errors and data inconsistency, so you may want to customise both folders location. 

To do so navigate to the Admin app -> Disk manager page ( /_admin_/diskmanager/ ) and change the disk allocations settings, specifying the desired size and location (absolute path).

Make sure the Workflow app temp and MonetDB Data Engine are in different folders, and in a location with sufficient disk capacity to include the total fixed sizes configured.

After entering new values here:

1. Click "Save changes"; the changes will be shown but will not be in effect yet
2. Make a note of the previous location, shown in the Disk Manager page under "Omniscope Disk status" section. For example: C:\Users\username\AppData\Local\Visokio\Omniscope\MonetDb
3. Shut down the Omniscope server process (e.g. from right-clicking blue system tray icon on Windows)
4. If you have production reports, you should migrate the data for the MonetDB Data Engine at this point, from the old location (from step 2) into the new folder configured (if you omit this step, report data will be automatically rebuilt when the report is next opened, with a small delay for each report).
5. Start Omniscope
6. Check the main log in /_admin_/logs that MonetDb started and DB farm is correctly chosen
7. Check again the Disk Manager page /_admin_/diskmanager/ is showing correct status.
   
   

2. Change Workflow Data compression

When Omniscope executes your data workflows, the temporary and persisting data for each block in your workflow is stored next to your project file in a ".data" folder.

By default Omniscope does not compress the workflow data, to favour write performance though requiring more space.

You can control this by deciding whether to apply a "Fast" compression, so data will be fairly compressed while still enough fast to write, or choose "High" when you want instead to have the maximum data compression to save disk space sacrifying write performance. 

Navigate o the Admin app -> Settings > Advanced page ( /_admin_/advanced/ )  , you'll find the option in the Workflow section.

3. Change dormant projects setting

Omniscope stores execution data for blocks in the project's ".data" folder, and this can be seen by the block appearing to be "full", visually. This is beneficial when actively working on a project and repeatedly executing, since it avoids the need to repeat the work done in the previous execution.

However over time this can build up. For projects no longer being actively worked on or executed, Omniscope 2021.1+ will periodically delete this non-essential data. (Note that essential data - for blocks directly connected to a report - will never be erased.) 

This non-essential data will be deleted for all projects in the sharing folder, that are considered "dormant". Omniscope  considers projects "dormant" if they have not been updated or executed within a configurable period, default 1 week.

If you are using a version prior to 2021.1, you should upgrade, since earlier versions do not manage this aspect of disk space unless the disk becomes critically full.

Otherwise, if needed, you can make Omniscope clean up data earlier than the default of 1 week, by adjusting the "Dormant projects" setting in Admin app > Settings > Advanced > Project data clean-up.