Update: An earlier version of these Release Notes gave an incorrect Docker Repository Image tag. The image tags should be the version number only (e.g. "1.6.0"). The documentation below has been corrected. We apologize for the error.

We're pleased to announce the immediate availability of Senate Matching Release 1.6.0 (October 2019). This release changes the default token format to use hexadecimal notation. All current Senate Matching customers are required to update their Contributor Nodes to 1.6.0 prior to their next matching project.

Here's what you need to know:

  • About the change: Senate Matching Contributor Nodes generate random, 64-bit integers. Prior to 1.6.0 these have been displayed as decimals (e.g. 45583393508801086). We've had feedback that some tools try and convert these strings into integers which occasionally causes problems (e.g. Excel will only keep the first 15 significant digits). So from 1.6.0 we're displaying tokens in a hexadecimal format (e.g. our token above would become "0xde0d9c10d585b4"). 
  • Impact and Risk: Customers have indicated that they already treat tokens as strings, in which case the impact minimal. Our testing with common data tools (Excel, Python, and R) has been positive.  
  • Actions: You will need to (1) upgrade your Contributor Node (see below); (2) Download your token mapping file; and (3) Upload new tokenized attribute data to Senate. Alternatively, you can use the Senate View Builder to convert your decimal tokens to hexadecimal format. Contact support@datarepublic.com for instructions. 

Downtime: The release is scheduled to go to production environments Thursday, 31 October 2019 (Australian time). System downtime is expected with this release, for up to one (1) hour during the release window. The release windows are: 

  • USA: 3:30pm-6:30pm Wednesday 30th October PST 
  • Australia: 2pm-5pm, Thursday 31st of October  AEDT
  • Singapore: 5am-8am, Friday 1st November SGT 

Here's what you need to do:

All customers are encouraged to upgrade to the new release at their earliest convenience.

To upgrade your Contributor Node, follow these steps:

1. Shutdown your current Contributor Node

To stop your currently running node, use the command contributor.sh down which will cleanly shutdown the Docker container.

$ sudo bash contributor.sh down

2. Update your start up script

Edit your version of contributor.sh to point to the latest Docker image tags. The relevant lines are given below (edit your file to match this):

contributor.sh (example)

export HITCH_DOCKER_IMAGE_TAG="1.6.0" # latest (Oct 2019)
export HITCH_UI_DOCKER_IMAGE_TAG="1.6.0" # latest (Oct 2019)

3. Restart your Contributor Node

Start your Contributor Node with the contributor.sh up command. The script will download the latest version of the node software from our repository. Your data will be preserved during the update. The "-d" option starts the Docker containers in the background. Depending on your local environment, you may or may not need to run as sudo.

$ sudo bash contributor.sh up -d

FAQ

Do I HAVE to update?

Yes. Senate will no longer understand how to perform token masking on decimal-encoded tokens. You will not be able to load match results into a Workspace until your tokens are in hexadecimal format.

Customers who have already loaded packages with decimal tokens into Senate can make use of the View Builder to create a view which will encode the old decimal tokens into hexadecimal. Contact Customer Success for detailed instructions.

Where do I get contributor.sh from?

Use your previously distributed version. Contact Customer Success if there are any issues.

How do I know what version I am running?

Use your browser to visit your Contributor Node UI. You do not need to log in. Check the bottom of the web page:

  • If you see a message starting "Version" (e.g. "Version 1.6.0 (build g13b5da0)") then this is the current version of your Contributor Node.
  • If you only see a copyright message, then you are running version 1.5.0 or earlier and should update.

What security / testing / review checks have been performed on this release?

  • QA performed a full regression test and tested performance of token databases containing up to 20M records. Recommended max database size is 10M rows.
  • Static code analysis (Gosec) scanned 163 files and found 0 errors or warnings.
  • All code is peer-reviewed in a "pull request" by at least one other developer before it is accepted onto the release branch.
  • All (automated) unit tests passed.
  • All (automated) integration tests passed.

What else has changed in this release?

  • Performance improvements to the Aggregator Node for large matching requests (10-20M rows). 
  • Some bug fixes for error reporting when an Aggregator Node fails to get a match (e.g. because no data has been loaded for one of the token databases).
Did this answer your question?