This article provides an overview of the end-to-end process for data sharing and how different work streams may be managed.
Note: If your project involves Senate Matching, please refer to the Senate Matching Project Guide as there are additional steps involved, otherwise, please continue below.
- You have received your Senate login
- You have reset your password upon first login
There are 2 main work streams for data sharing on Senate:
- Data preparation: where data is uploaded and prepared for projects (using the Manage Data menu)
- Project creation (and management): where organizations can collaborate and manage their data sharing (using the Project menu)
The below diagram illustrates the 2 main work streams and any dependent steps:
Note: a data package must first be created before it can be added to a project.
- Data preparation can be started by each organization independently to Project Creation.
- Project creation (and management) is facilitated by the organization who is providing the Workspace.
Data Preparation Work Stream
Note: Skip this work stream if you will not be contributing data to the project.
In this work stream, each organization focuses on loading their data into Senate, and creating packages for Projects.
0. Data Preparation (completed in your organization's environment):
- Refer to your Project Manager for more information on what data needs to be prepared.
- Make sure data is cleaned (i.e. no duplicate rows, removal of null value rows, standardized formatting for fields, etc.) to ensure that the analysts can analyze the data as soon as possible.
- Data should be prepared according to what has been agreed between organizations.
1. Upload Files: Drag and drop files less than 100MB or use SFTP for larger files.
2. Create Database: If you would like to query structured data (using View Builder) or allow approved analysts to query your data in a Redshift database, you will need to create a database.
4. Create a View of your table using View Builder (optional): This is optional, in the instance where you would like to share only a subset of your data for specific projects.
5. Create a Package: Specify the tables and files that you would like to contribute to a Project. A package contains meta-data only (i.e. file/table names).
Project Creation Work Stream
This is where you collaborate with other organizations for a Project.
1. Add People: Add people who are needed for the Project (who have Senate user accounts).
2. Have Conversations: Use the Conversations to converse with everyone in the Project, or create subgroups (i.e. your organization and DR) to have all communications in one place.
3. Select a Legal Framework: Participating organizations can use Data Republic's Common Legal Framework for data exchange (if this has been executed by parties involved) or create and select their own legal framework to use in Projects (if an organization has signed the Third Party Module).
4. Add Packages (from the Data Preparation work stream). If you will be contributing data to a Project, you will need to add Packages to the Project (to specify which files and/or tables you will be sharing).
5. Submit the Data License: A Data License should be drafted while the data is being prepared, as organizations will need to allocate enough time to discuss, negotiate, agree on terms for permitted use of data and approve the data license.
Once data analysis is underway in the Project, the analyst will explore the data and create any outputs if the Data License allows for this. Upon project completion, the Workspace will be terminated.
1. Analyse Data and Request Outputs (if allowed by the Data License): Only authorized users from organizations permitted in the Data License can access data for analysis and request output extraction. Output extract requests will be reviewed by the Senate Platform Administrator for approval and will be rejected if it does not align to Data License terms.
2. Terminate Workspace: Once the Project is complete and permitted outputs have been extracted, the Workspace can be terminated, and all of the data inside deleted.