Omniscope projects release management

Omniscope provides an easy and comprehensive set of tools that allow you to manage projects in your organisation across development and production servers.

This article discusses how you can setup Omniscope to manage the deployment of projects, workflows and reports.

Key features and vocabulary:

• Working copy - a Working copy allows you to work on a project on a separate server or within the same server in a different folder on a copy of an original project. Once you are happy with the changes you can then publish/push those changes to the original project. Or revert the changes and pull from the original project.
  
  
• Project Version History - see workflow and report changes over time, who authored what , and work with confidence. Revert or create a copy out of a specific revision.
  
  
• Scheduler - Provides an interface to automatically schedule certain actions and task on a regular basis or based on events.
  
  
• OpenID - Omniscope provides ability for you to authenticate using OpenID providers such as Google, Okta, KeyCloak, Auth0, EntraID, to authenticate your users.

We recommend that you have at least 2 servers. By server we mean a dedicated machine (Linux recommended). You could potentially have a third-server which is a test server where you can deploy new builds to test everything works before deploying to production. For system requirements see here.

From 2024.2 Rock release we introduced new features to complement the functionality and bring further improvements.

• Working Copy actions (Push/ Pull/ Detach) for scheduler tasks will orchestrate synchronisation between the staging and production environments
• Multi-Project action: Execute tasks on multiple projects inside a target folder (e.g. refresh 100 customers’ projects with one task!).
• Warm-up reports" action will ensure your data is querying-ready for optimised performance.
• App-wide parameters in the scheduler task configurations allow you to centrally manage setting switches in multiple locations (e.g. change the publishing folder)

Production server

This server acts as a production server. All your end-users will use this server to view the final projects/Reports.

The scheduler on the production server can also be used to automatically refresh the projects/reports on a regular basis.

Since this is the customers facing server, it is on this server you would setup authentication for your end-users using OpenID which is highly recommended.

You can also use the "Admin dashboards" to audit who is accessing which Reports and general server performance.

You would need to allow your project editors to have "Project publisher" permission on the production server so that they can create working copies on the development server. For more information on permissions see here.

The production server should also have resources access setup to manage what local resources should be accessible by projects.

Omniscope also has a concept of virtual roots which allows you to create shortcuts on the main project list page so that your end-users can easily navigate to specific folders.

It is highly recommended you enable HTTPS on this server. For information on how to do this see here.

If your projects use custom blocks you should configure your server from Admin web app > R/Python and Blocks library to run in Docker for added security.

Development server

This server acts as a development server. You project editors would typically use Working copies to work and deploy projects, workflow and reports to the production server. 

You can additionally use Scheduler actions , specifically multi-project and working copy actions to automatically deploy the changes to the production server in bulk. You can also use the Scheduler to periodically backup the projects on both the production or development servers.

When working on a project, whether you are authoring a workflow or a report, Omniscope keeps track of the changes and allow you to see the project version history, so you can understand who made changes - especially when collaborating on the same project,  revert to a previous version or make a copy out of a specific version.

NOTE: Project history and data are automatically deployed to the production server when you use working copy actions as behind the scenes its the same as you downloading the IOZ from the development server and then importing it in production server.

This server doesn't need to be accessible to the outside world and should be deployed within your company VPN so that only people within the organisation have access to it. However, you should configure permissions accordingly to ensure that project editors have relevant access. 

Project editors

The project editors who are responsible for creating the projects and managing them will normally be working on the development server by creating new projects and/or updating existing ones. 

Working copy functionality requires the project to be already present on the production server. So when a new project needs to be deployed. It would first need to be created on the development server and then manually deployed to production by downloading it as ioz and re-importing it onto the production server.  Note this is only needed first time after that you can create working copy of the original project on the production server onto the development server.

We highly recommend you use project parameters for configuring your database connections and other settings so that you can easily change from production database to a test database on a development server. 

Your project editors can utilise the project templates to create a set of pre-configured projects for rest of the team to use to create new projects.

End-users

End-users (your customers) will be accessing the production server, they should be configured with the relevant permissions. In most cases these are likely to be List directory and Shared Report viewer permissions.

Working copies in practice: staging to production workflow

When you edit a project directly, your changes are immediately visible to anyone viewing that project’s reports. For minor updates this may be fine, but for larger or potentially disruptive changes — restructuring a workflow, changing data sources, redesigning reports — you may want to work in isolation until the changes are ready. Working copies provide this isolation: you work on a linked copy of the project, and only when you are satisfied do you push the changes back to the original. Viewers and consumers of the original project are unaffected until you choose to publish.

This section walks through the practical steps for setting up and using this workflow.

Working copies are not limited to cross-server scenarios. While a common setup is to have separate development and production servers, you can also use working copies within the same server — for example, keeping a staging copy in one folder and the published original in another. The workflow below applies equally to both approaches; wherever it refers to “development” and “production”, you can substitute any two locations that suit your organisation’s process.

Initial setup

1. Build the initial project. Author your project — workflows, reports, data connections — in a local Omniscope installation or any environment separate from production. This could be a dedicated development server, a personal laptop, or a staging folder on the same server.
2. Deploy to production for the first time. Download the finished project as an IOZ file and import it onto the production server (or into the production folder). This manual step is only needed once per project — after that, working copies handle all subsequent deployments.
3. Configure permissions. On the production server, ensure your project editors have the Project Publisher permission. This allows them to create working copies linked to the production originals. If you want to prevent direct edits to production projects, turn off the Project Editor permission and rely solely on Project Publisher — this way, changes can only reach production through the working copy workflow.
4. Create a working copy. On the production server (or in the production folder), navigate to the project, click the three dots menu, and select “Make a Working copy”. Copy the link it provides, then in your staging environment click “Create a working copy here” and paste the link. See the Working Copies article for the full step-by-step with screenshots.

Day-to-day editing cycle

Once the working copy link is established, the typical workflow is:

1. Edit the working copy in your staging environment. Make changes to workflows, reports, data connections — whatever is needed. The original project in production remains untouched.
2. Review and test. Verify your changes work as expected. Use project parameters to point at test databases or sample data sets so development work doesn’t affect production data.
3. Push to production. When ready, open the working copy and click Push. This replaces the original project with the working copy. Project history and data are included — behind the scenes it is the same as downloading the IOZ and importing it into the production location.
4. Pull updates if needed. If another editor has pushed changes to the production original since you last synced, you can Update/Revert your working copy to pull those changes down before continuing your own edits.

Automating with the Scheduler

From 2024.2 onwards, you can automate the working copy workflow using Scheduler actions rather than doing it manually:

• Working Copy Push/Pull/Detach actions — schedule automatic pushes from staging to production at set times (e.g. nightly, after business hours), or pull the latest production state into your staging environment on a regular basis.
• Multi-Project actions — execute tasks across an entire folder of projects at once. For example, if you have 50 customer projects that all need refreshing after a deployment, a single multi-project action can handle them all.
• App-wide parameters — use centralised parameters in scheduler task configurations to manage settings like the target publishing folder. Change the parameter once and it takes effect across all tasks that reference it.

A common automated pattern is:

1. Editors push changes to working copies throughout the day
2. A scheduled task runs each evening to push all working copies in a folder to production
3. A follow-up task refreshes the production projects’ data
4. A warm-up reports task ensures report data is query-ready before users arrive in the morning

NOTE: OpenID Connect authentication is not currently supported when using the Scheduler tasks to autonomously push/pull working copies. If your servers use OpenID for user authentication, the automated working copy Scheduler actions will not be able to authenticate. In this case, push and pull operations need to be performed manually by a logged-in user.