Status Updates

Weekly Development Report

CORE

• Exchange Onboarding

In the past week almost all ported V1 endpoints have been been coded, manually tested and merged into the develop branch. The proposal for testing exchange endpoints (v1) is also almost finished. It is still in need of a written example as to how to test a V1 endpoint.

• Core Backlog

Several key team members discussed the subject of refactor exception handling in CSL. It was voiced that a change to the guidelines could have big implications on how the process is handled. One of the team members voiced concerns about the proposed process, however, a solution encompassing that team member’s concern was proposed and is now being flushed out further. The HTTP-based block sync proof-of-concept feature consists of several components: Block downloading, which is ready and is currently being tested, the Blockchain dump creation tool, which is also now ready and is now awaiting review. Taking agreed upon deadlines into consideration, the team feels that they can launch without it, as blockchain dumps can be created manually and they only need to be created once a week. Furthermore, Daedalus integration is in progress. The team reports that it will not take much time, however, smoke testing will be required. Hosting of the blockchain dump can be done in a rather short amount of time, but such an operation would require the assistance of DevOps.

• API Rewrite

Transaction end points (V1) have been successfully implemented and tested with the same logic as V0.

DAEDALUS

During the week the Daedalus team performed manual testing and quality assurance tasks for the upcoming Daedalus 0.9 and Cardano 1.1 release. During this process, some smaller, mostly cosmetic and user experience issues were detected and fixed.

• Log Submission

A significant effort was applied to implement functionality which would allow users to send bug reports directly from Daedalus. Two more flags (--store, --send-logs) have been added to the code base. Currently, the report server only accepts a maximum of one file. In the past week, the maximum file size upload increased to 25MB. This functionality is now being tested.

WALLET BACKEND

• Wallet backend data layer rewrite

A team member has completed his initial assignment in generating synthetic blockchains via QuickCheck. A different team member has began to write the actual wallet logic code starting by block application. Unfortunately, the progress has been slowed due to ramp up time. Yet another team member is tackling the specification for block rollbacks, which are then going to be reviewed by a senior engineer and implemented by the team.

• Work on current wallet backend

Performance measurements are scheduled to resume immediately after the team member, previously assigned to the task, is finished with 1.1 release testing. A new benchmark was added for the New Wallet endpoint. Mempool consistency has been completed and merged, the DB schema change and migration has also been completed and is now awaiting to be merged. The migrations are being tackled as part of the benchmarking effort. A team member was focused on trying to benchmark his changes to "prove" the remedial work was effective. The team plans to keep working on the benchmarks and to backport previous work done by a team member, working on this task, to the develop branch.

NETWORKING

Multiple pull requests have been completed and are now ready and/or are being reviewed. The Implement (block) verification proposal is currently blocked by a different pull request. However, two of the team’s members are working on and expect to be done by Wednesday. After that task has been successfully completed, a team member will then be able to complete the concurrent block streaming/verification.

DEVOPS

In the past week, the team has made significant progress on the Linux installer. Currently they are still waiting on the best solution to package NPM with Nix. Additionally, the team made several improvements to our CI as well as supporting development for the upcoming release.

MISCELLANEOUS WORKSTREAMS

• Hardware Wallets

The UAT part of Phase 1 has been planned, with a view to the Phase 1 release candidate being completed and delivery sign off being issued to a blockchain consultancy firm at the beginning of March. The planning for Phase 2 (the integration of Ledger Nano S functionality with the Cardano wallet) is also scheduled for the beginning of March. The production of the specification for the Daedalus implementation has been delayed due to testing work for the February release of Daedalus. It has been agreed that the initial release will not support delegation. A team member continues to work with the blockchain consultancy firm, on resolving questions around the ED25519 derivation implementations used by Ledger and IOHK.

• Incentives

Two of the researchers were expected to deliver the solution proposal by 16 February, 2018. However, they have requested a further extension until the end of February to refine the paper. The meeting planned for the middle of February 2018 to review the proposal (and move to the technical specification and simulation phase) has been rescheduled for the beginning of March. This is the latest it can be pushed before it starts to block planned progress on Decentralisation. Once this occurs, the incentives scheme will be evaluated before the process of planning its implementation and integration into the Cardano platform. Two of the senior engineers will execute the simulations needed to establish the required level of confidence in the model.

• Smart Contract Language

Recruitment for roles has been promoted and is in progress. Two candidates referred internally are going through selection. A new member has joined the team and is taking part in design discussions with two senior engineers. Work started on building the test suite to validate language functionality. The discussions are currently ongoing.

• Sidechains

Fundamental research has been completed and a detailed academic paper developed by the international research team reporting to the chief scientist. The team are looking at potential conference submissions, which could require some empirical analysis. Paper to be shortly distributed to engineering for internal review.

Tech Support

Last week did not see a significant variation in the number of received tickets. The tech support team received a total of 475 tickets, out of which 198 were successfully resolved and the remaining 277 are being followed up. The top reported issues were:

• Unable to connect to network
• Antivirus-related issues
• Cannot launch
• Transaction-related questions

Weekly Development Report

CORE

This week the team has somewhat shifted focus from regular development to contribute to writing test cases for release 1.1.0. Testing of 1.1.0 will be one of the team’s major objectives for the next few weeks.

• Block syncing

Work on block application optimization has steadily progressed. This will allow for gradual performance improvement of block syncing. Furthermore, an experimental effort is being taken to have proof-of-concept for HTTP mirroring of block syncing. This is almost finished and the code is currently in the integration testing stage.

• Refactoring

Exception handling refactoring has steadily progressed over the past week. The team is approaching the goal of removing all exception handling anti-pattern occurrences to make the code comply to the recently agreed exception handling guidelines. Some work is currently being done on eliminating bad concurrency usages (which is a step forward aimed at a reliable shutdown for the Cardano node). Also, code restructuring refactoring has been successfully completed (the last remaining work was about moving test modules to their proper place).

• Documentation

Several internal documents were finally agreed upon and merged. These documents include a detailed description for the update system, exception handling guideline and a code diagram for the Cardano SL repository. Work is currently underway to provide detailed documentation on block processing. Last, but not least, specifications for machine-readable logging and efficient storage are being improved.

DAEDALUS

The main goal of the week was the preparation of the next major release 0.7 for React Polymorph library used in Daedalus and testing all of the breaking changes introduced there against Daedalus. The remaining time was spent on testing for the upcoming Daedalus 0.9 and Cardano 1.1.0 releases.

WALLET BACKEND

As of now, the instructions have been sent to the exchange and the migration is pending. A team member will migrate the code to the Master branch so that the team can decommission the prior code. The exchanges tech team has requested more work from our side.

• Benchmarking

As of last week, the Baselining has been successfully completed. A key member of the team is working on a tool (“dbgen”) to generate synthetic data for the wallet backend, to be used by the benchmarks. As for ongoing work, the second round focusing on working with real-world data has begun.

• API Rewrite

A team member has been specifically assigned to assist with manual testing. For a more smooth transition process, the old and the new API will coexist (at least initially) on the same wallet backend, making integration easier.

• Data Layer Rewrite

Version 5 of the semi formal specification has been completed. It has been decided that an extension will be created to include block rollbacks, transaction input selection, and miscellaneous database issues. Last week, partial implementation was initiated and a team member is currently writing the wallet’s business logic. A significant portion of the testing framework has been successfully put in place (Code that translates from simplified descriptions of transactions and blocks into real data types).

NETWORKING

The team is awaiting sign off on new requirements from the stakeholders. The work to decouple the networking layer from the core logic has nearly been completed. The networking team continue to be dedicated to the code update project. Some of the efforts there relate to the networking requirements for decentralisation, therefore the work will also benefit the Shelley programme.

DEVOPS

The team focused on supporting the 1.1 release testing and deployment. Some further work was done on automating the releases and finalizing migration of current CI to the new integration pipeline management software. The team has also implemented code-as-infrastructure for the upcoming recovery service.

MISCELLANEOUS WORKSTREAMS

• Sidechains

The team is working towards submitting a paper to a conference by the 13th February 2018. The researchers had some exploratory discussions regarding the implementation phase but details are deferred until after the submission date.

• Hardware Wallet

The UAT part of Phase 1 is in planning, with a view to Phase 1 being completed and delivery sign off being issued to a blockchain consulting company in the first week of March. Planning for Phase 2 (the integration of Ledger Nano S functionality with the Cardano wallet) continues to be delayed due to resource unavailability. A product manager is producing specification for the Daedalus implementation which is scheduled for review on the 15th of February (although this is now highly likely to be delayed). It has been agreed that the initial release will not support delegation.

Tech Support

Last week saw yet another drop in new tickets. The tech support team received a total of 415 tickets, out of which 134 were successfully resolved and the remaining 281 are being followed up. The top reported issues were:

• Unable to connect
• Various transaction questions
• Issues with using the wallet
• Antivirus-related issues
• Cannot launch

Weekly Development Report

CORE

The team concentrated on Exception Handling, which is 70% complete and due to be finished shortly. Reliable Shutdown will be finished by the end of the sprint. Requirements have been gathered for the Revised Monad Stack and are due to be scheduled for the next sprint. Code Restructuring is 80% finished. Improving the structure of the testing module remains to be done.

DAEDALUS

The main goal of the week was the preparation of the 0.9.0 Daedalus release, which will be shipped with Cardano 1.1. One of the bigger tasks was also finalized, refactoring the build system to use Gulp 3, which solved a lot of issues introduced with the upgrade of the WebPack to version 2. Work on wallet account features continued.

WALLET BACKEND

The team put a lot of time into preparation for the exchange code migration. The instructions on the pending migration have been sent to the exchanges. Data Consistency has been merged to Master. The team is pushing to have Targeting Dev complete shortly. A performance improvement for Multi-Action Output has been merged to Master. The team is currently deciding whether other performance improvement initiatives are required. A phased approach is planned once the Data and Performance workstreams are completed and as improvements become available. Additionally, Baselining has also been completed successfully. The team has successfully delivered a prototype of the Exchange Simulator (Request Generator) for ongoing tests and performance improvements. Currently, a team member is working on a tool (“dbgen”) to generate synthetic data for the wallet backend, to be used by the benchmarks. Various other tasks regarding the wallet backend have been simplified or integrated. A key team member will give new estimates on task completion. Two of the team members will concentrate on manual testing of the API associated with the tasks. The old and the new API will coexist (at least initially) on the same wallet backend, making integration easier. The Daedalus team can start integrating the new API, and can do this in a “cherry-picking” fashion. There is no need to commit to the whole V1 API just yet. A key team member has been designated as the squad lead currently working on this, incorporating the specification from another team member and translating it into an “executable specification” for the new wallet. A senior engineer is working on a semi-formal specification that will shape the data layer and its storage requirements. Partial implementation has started and a significant portion of the testing framework is in place (code that translates simplified descriptions of the transactions and blocks into real data types). A team member has started working on the generation of synthetic test data. Another key member is working on plugging the wallet backend to this new data layer.

NETWORKING

The team focused on the relay connection timeout, which will be included in the next release. Streaming block syncing is a dependency for Deadalus to provide a block syncing progress bar with smooth (rather than jumpy) reporting.

DEVOPS

Last week the DevOps team coordinated branch off in preparation for the Cardano SL 1.1 release. The first pass of the staging 1.1 deployment was completed with one issue found in relation to moving the CI to a new provider. Further highlights of the week have been a successful release of stack2nix, as well as work with exchanges on implementing efficiency and scalability improvements.

MISC. WORKSTREAMS

• Smart Contracts Language

The stakeholders have had detailed discussions about deliverables for the workstream of a new language designed to make it easier to create smart contracts. A team member is working on the language core and language initiative (specification, presentation and the tutorial for developers, which are prerequisites for this workstream).

• Sidechains

Research is progressing with a view to presenting the findings in a formal academic context in the near future.

Tech Support

There was a decrease in the number of new tickets last week. The tech support team received a total of 593 tickets, out of which 308 were successfully resolved and the remaining 285 are being followed up. The top reported issues were:

• Unable to connect
• Various transaction questions
• Issues with using the wallet
• Antivirus-related issues
• Cannot launch

Weekly Development Report

CSL CORE

Here are the latest updates from the Core team:

• Cardano Launcher now takes JSON config as a parameter. Thus, no system-specific launch scripts are needed (previously, the Launcher required a lot of parameters to be passed).
• The recent fixes from master were merged to develop.
• Exception handling refactoring is in progress. This will improve the safety and predictability of the code by removing error-prone patterns.
• The team is working on the reliable shutdown of a node. A lot of the code was recently refactored, a few occurrences of bad concurrency patterns were removed.
• A few optimizations have been applied to block syncing (this is work in progress).
• An experimental prototype for HTTP-based block syncing is almost ready. Once completed, this should allow expedited block syncing.
• A few optimizations were performed for “high traffic” wallet users (e.g. exchanges), in particular, address generation is now >100x faster (the result of wiping out several data access anti-patterns).

DAEDALUS

The Daedalus team focused on Byron-related tasks. In the scope of this effort, numerous improvements were built for the upcoming release of the Daedalus Cardano wallet. This includes a feature that detects cases of a Cardano node being out of sync due to internet connectivity issues and reports this to the user by displaying the “Connecting” screen with the appropriate message, moving the user away from the main user interface. A New “About” screen has been implemented. Now it shows bot Daedalus and Cardano client versions for Cardano, and the Mantis client version for Ethereum Classic. Furthermore, a bug regarding the incorrect detection of time on the user's machine being not in sync with the global time was detected and fixed. This removed the false-positive report which prevented some users from using the wallet when their time was only slightly off. The Electron platform running Daedalus has been updated to resolve a security flaw in Electron.

MIDDLEWARE

Here are the latest updates from the team:

• A team member is to resume working on the new wallet V1 API, tackling the endpoint to calculate the fees for a transaction.
• The team is focused on performance improvements and stability of the wallet backend.
• Two key members are to continue working on the benchmarks and will move to stress tests soon afterwards.
• Another team member will extend "dbgen" (a tool to generate synthetic wallet-db data) to generate fake history entries. This synthetic data will be used in the benchmarks to simulate realistic workloads.
• The updates on consistent DB updates for the old wallet are to be reviewed and merged.
• A team lead is to continue fleshing out the formal specification for the new wallet with an eye on asymptotic complexity.
• Two team members have converged on the same specification for UTXO accounting, so that it can drive the implementation of the data layer.
• A team member is to implement a solution for a much faster procedure to restore an HD-wallet from a seed.

NETWORKING

A small but important change that increases the network connection timeout from 2 to 15 seconds has been merged to the development branch. The team expects that this will eliminate certain connection problems (2 was too low to begin with). They continued design work on a better block download system that will make Daedalus faster to start up. Work on supporting features for a better UX for Daedalus initial sync hasn't progressed yet because its developer is away teaching Haskell in Barbados). Those two pull requests have been moving along: devs working on the issues are tying up loose ends and nearing approval from the core developers. They are optimistic about having them merged by the end of the week.

DEVOPS

This week's update will be available shortly.

Tech Support

Last week saw a bit of an uptick of new tickets. The Tech support team received a total of 1214 tickets, out of which 828 were successfully resolved and the remaining 386 are currently being followed up and resolved by the tech support team. The top reported issues were:

• Unable to connect
• Unable to see transactions
• Issues with using the wallet
• Antivirus related issues
• Cannot launch