diff --git a/docs/MIGRATION_GUIDE.md b/docs/MIGRATION_GUIDE.md deleted file mode 100644 index f1a8267..0000000 --- a/docs/MIGRATION_GUIDE.md +++ /dev/null @@ -1,162 +0,0 @@ -# Migration Guide from Arweave's SmartWeave SDK to Warp SDK - -This guide describes the simplest way to switch to the new version of SmartWeave. It uses `WarpFactory` for Web to quickly obtain fully configured, mem-cacheable SmartWeave instance. To see a more detailed explanation of all the core modules check out the [source code.](https://github.com/redstone-finance/warp) - -### You can watch this tutorial on YouTube 🎬 - -- [Youtube link](https://www.youtube.com/watch?v=fNjUV7mHFqw) - -[![warp migration guide](https://img.youtube.com/vi/fNjUV7mHFqw/0.jpg)](https://www.youtube.com/watch?v=fNjUV7mHFqw) - -### Need help? 🙋‍♂️ - -Please feel free to contact us [on Discord](https://redstone.finance/discord) if you face any problems. - -## 1. Update dependencies 📦 - -#### 1.1 Install Warp - -```bash -# Yarn -yarn add warp-contracts - -# or NPM -npm install warp-contracts -``` - -#### 1.2 Remove smartweave v1 - -If smartweave was installed globally, add `-g` flag to npm or use `yarn global` - -```bash -# Yarn -yarn remove smartweave - -# or NPM -npm uninstall smartweave -``` - -#### 1.3 Replace imports - -You can import the full API or individual modules. - -```typescript -import * as WarpSdk from 'warp-contracts'; -import { Warp, Contract, ... } from 'warp-contracts'; -``` - -## 2. Update your implementation 🧑‍💻 - -### 2.1 Initialize a Warp client - -```typescript -import Arweave from 'arweave'; -import { WarpNodeFactory } from 'warp-contracts'; - -// Create an Arweave instance -const arweave = Arweave.init({ - host: 'dh48zl0solow5.cloudfront.net', - port: 443, - protocol: 'https', - timeout: 20000, - logging: false -}); - -// Create a Warp client -const smartweave = WarpNodeFactory.memCached(arweave); -``` - -In this example we've used the `memCached` method. You can see other available methods in documentation: - -- [For Web](https://smartweave.docs.redstone.finance/classes/SmartWeaveWebFactory.html) -- [For Node.js](https://smartweave.docs.redstone.finance/classes/SmartWeaveNodeFactory.html) - -#### [Optional] Custom modules 🛠 - -Warp has a modular architecture, which allows you to connect custom modules to any part of the Warp client implementation. See [custom-client-example.ts](https://github.com/redstone-finance/redstone-smartweave-examples/blob/main/src/custom-client-example.ts) to learn more. - -### 2.2 Initialize contract object - -```typescript -// Simple connection (allows to read state) -const contract = warp.contract('YOUR_CONTRACT_TX_ID'); -``` - -💡 Note! For being able to write interactions to blockchain you need to connect wallet to contract object. - -```typescript -const contract = warp - .contract('YOUR_CONTRACT_TX_ID') - .connect(jwk) // jwk should be a valid private key (in JSON Web Key format) - .setEvaluationOptions({ - // with this flag set to true, the write will wait for the transaction to be confirmed - waitForConfirmation: true - }); -``` - -### 2.3 Interact with your contract - -#### Read state (readContract in V1) - -```typescript -// Read state (similar to the "readContract" from SmartWeave V1) -const { state, validity } = await contract.readState(); - -// state is an object with the latest state - -// validity is an object with valid and invalid transaction IDs -// E.g. { "TX_ID1": true, "TX_ID2": false, ...} -``` - -#### View state (interactRead in V1) - -```typescript -// View state (similar to the "interactRead" from SmartWeave V1) -const { result } = await contract.viewState({ - function: "NAME_OF_YOUR_FUNCTION", - data: { ... } -}); -``` - -#### Write interaction (interactWrite in V1) - -```typescript -// Write interaction (similar to the "interactWrite" from SmartWeave V1) -const result = await contract.writeInteraction({ - function: "NAME_OF_YOUR_FUNCTION", - data: { ... } -}); -``` - -💡 You can read detailed explanation of each contract method [here.](https://github.com/warp-contracts/warp#contract-methods) - -### [Optional] 2.4 Confgure logging - -Warp uses `tslog` library for logging. By default logger is set to "debug" level, which means that all messages with level "debug" or higher are logged. - -#### Update logger options - -```typescript -LoggerFactory.INST.setOptions({ - type: 'json', - displayFilePath: 'hidden', - displayInstanceName: false, - minLevel: 'info' -}); -``` - -Learn more about available logging options in [tslog documentation.](https://tslog.js.org/tsdoc/interfaces/isettingsparam.html) - -#### Update logger level - -Instead of updaitng all logger options you can simply set the new minimum logger level - -```typescript -LoggerFactory.INST.logLevel('info'); -``` - -Available log levels are listed [here.](https://github.com/redstone-finance/warp/blob/main/src/logging/RedStoneLogger.ts#L1) - -## 3. Test everything 🔥 - -Before deploying your changes test it carefully. If you face any problems please contact us [on our discord](https://redstone.finance/discord). We'll be happy to help 😊 diff --git a/package.json b/package.json index c327f42..f9c1e35 100644 --- a/package.json +++ b/package.json @@ -119,6 +119,7 @@ "vm2": false, "archiver": false, "stream-buffers": false, - "constants": false + "constants": false, + "knex": false } } diff --git a/src/__tests__/integration/basic/arweave-transactions-loading.ts b/src/__tests__/integration/basic/arweave-transactions-loading.ts index a2d4107..fc20367 100644 --- a/src/__tests__/integration/basic/arweave-transactions-loading.ts +++ b/src/__tests__/integration/basic/arweave-transactions-loading.ts @@ -55,7 +55,7 @@ describe('Testing the Arweave interactions loader', () => { await addFunds(arweave, wallet); contractSrc = fs.readFileSync(path.join(__dirname, '../data/inf-loop-contract.js'), 'utf8'); - const contractTxId = await warp.createContract.deploy({ + const { contractTxId } = await warp.createContract.deploy({ wallet, initState: JSON.stringify({ counter: 10 diff --git a/src/__tests__/integration/basic/vrf.test.ts b/src/__tests__/integration/basic/vrf.test.ts index f6e579b..637de2a 100644 --- a/src/__tests__/integration/basic/vrf.test.ts +++ b/src/__tests__/integration/basic/vrf.test.ts @@ -7,7 +7,7 @@ import { ArweaveGatewayInteractionsLoader, defaultCacheOptions, EvaluationOptions, - GQLEdgeInterface, + GQLEdgeInterface, GQLNodeInterface, InteractionsLoader, LexicographicalInteractionsSorter, LoggerFactory, @@ -57,7 +57,7 @@ describe('Testing the Profit Sharing Token', () => { loader = new VrfDecorator(arweave); LoggerFactory.INST.logLevel('error'); - warp = WarpFactory.levelDbCached(arweave, { + warp = WarpFactory.custom(arweave, { ...defaultCacheOptions, inMemory: true }) @@ -150,24 +150,24 @@ class VrfDecorator extends ArweaveGatewayInteractionsLoader { async load( contractTxId: string, - fromBlockHeight: number, - toBlockHeight: number, - evaluationOptions: EvaluationOptions - ): Promise { - const result = await super.load(contractTxId, fromBlockHeight, toBlockHeight, evaluationOptions); + fromSortKey?: string, + toSortKey?: string, + evaluationOptions?: EvaluationOptions + ): Promise { + const result = await super.load(contractTxId, fromSortKey, toSortKey, evaluationOptions); const arUtils = this.arweave.utils; const sorter = new LexicographicalInteractionsSorter(this.arweave); for (const r of result) { - r.node.sortKey = await sorter.createSortKey(r.node.block.id, r.node.id, r.node.block.height); - const data = arUtils.stringToBuffer(r.node.sortKey); + r.sortKey = await sorter.createSortKey(r.block.id, r.id, r.block.height); + const data = arUtils.stringToBuffer(r.sortKey); const [index, proof] = Evaluate(key.getPrivate().toArray(), data); - r.node.vrf = { - index: useWrongIndex.includes(r.node.id) + r.vrf = { + index: useWrongIndex.includes(r.id) ? arUtils.bufferTob64Url(Uint8Array.of(1, 2, 3)) : arUtils.bufferTob64Url(index), - proof: useWrongProof.includes(r.node.id) + proof: useWrongProof.includes(r.id) ? 'pK5HGnXo_rJkZPJorIX7TBCAEikcemL2DgJaPB3Pfm2D6tZUdK9mDuBSRUkcHUDNnrO02O0-ogq1e32JVEuVvgR4i5YFa-UV9MEoHgHg4yv0e318WNfzNWPc9rlte7P7RoO57idHu5SSkm7Qj0f4pBjUR7lWODVKBYp9fEJ-PObZ' : arUtils.bufferTob64Url(proof), bigint: bufToBn(index).toString(), diff --git a/src/__tests__/regression/read-state.test.ts b/src/__tests__/regression/read-state.test.ts index 7bc2650..d32bc42 100644 --- a/src/__tests__/regression/read-state.test.ts +++ b/src/__tests__/regression/read-state.test.ts @@ -50,7 +50,7 @@ describe.each(chunked)('v1 compare.suite %#', (contracts: string[]) => { console.log('readState', contractTxId); try { console.log = function () {}; // to hide any logs from contracts... - const result2 = await WarpFactory.levelDbCached(arweave, { + const result2 = await WarpFactory.custom(arweave, { ...defaultCacheOptions, inMemory: true }) @@ -82,7 +82,7 @@ describe.each(chunkedVm)('v1 compare.suite (VM2) %#', (contracts: string[]) => { .readFileSync(path.join(__dirname, 'test-cases', 'contracts', `${contractTxId}.json`), 'utf-8') .trim(); console.log('readState', contractTxId); - const result2 = await WarpFactory.levelDbCached(arweave, { + const result2 = await WarpFactory.custom(arweave, { ...defaultCacheOptions, inMemory: true }) @@ -110,7 +110,7 @@ describe.each(chunkedGw)('gateways compare.suite %#', (contracts: string[]) => { async (contractTxId: string) => { const blockHeight = 855134; console.log('readState Warp Gateway', contractTxId); - const warpR = await WarpFactory.levelDbCached(arweave, { + const warpR = await WarpFactory.custom(arweave, { ...defaultCacheOptions, inMemory: true }) @@ -120,7 +120,7 @@ describe.each(chunkedGw)('gateways compare.suite %#', (contracts: string[]) => { const resultString = stringify(result.state).trim(); console.log('readState Arweave Gateway', contractTxId); - const result2 = await WarpFactory.levelDbCached(arweave, { + const result2 = await WarpFactory.custom(arweave, { ...defaultCacheOptions, inMemory: true }) @@ -142,7 +142,7 @@ describe('readState', () => { const result = await readContract(arweave, contractTxId, blockHeight); const resultString = stringify(result).trim(); - const result2 = await WarpFactory.levelDbCached(arweave, { + const result2 = await WarpFactory.custom(arweave, { ...defaultCacheOptions, inMemory: true }) @@ -166,7 +166,7 @@ describe('readState', () => { target: '6Z-ifqgVi1jOwMvSNwKWs6ewUEQ0gU9eo4aHYC3rN1M' }); - const v2Result = await WarpFactory.levelDbCached(arweave, { + const v2Result = await WarpFactory.custom(arweave, { ...defaultCacheOptions, inMemory: true }) diff --git a/src/contract/migration/MigrationTool.ts b/src/contract/migration/MigrationTool.ts new file mode 100644 index 0000000..4e38ba8 --- /dev/null +++ b/src/contract/migration/MigrationTool.ts @@ -0,0 +1,60 @@ +import { defaultArweaveMs, EvalStateResult, sortingLast } from '@warp/core'; +import Arweave from 'arweave'; +import { LoggerFactory } from '@warp/logging'; +import { LevelDbCache } from '@warp'; +import knex from 'knex'; + +export type MigrationResult = Array<{ contractTxId: string; height: number; sortKey: string }>; + +export class MigrationTool { + private readonly logger = LoggerFactory.INST.create('MigrationTool'); + + constructor(private readonly arweave: Arweave, private readonly levelDb: LevelDbCache>) {} + + async migrateSqlite(sqlitePath: string): Promise { + this.logger.info(`Migrating from sqlite ${sqlitePath} to leveldb.`); + + const knexDb = knex({ + client: 'sqlite3', + connection: { + filename: sqlitePath + }, + useNullAsDefault: true + }); + + const cache = await knexDb + .select(['contract_id', 'height', 'state']) + .from('states') + .max('height') + .groupBy(['contract_id']); + + this.logger.info(`Migrating ${cache?.length} contracts' state`); + + const result = []; + + for (const entry of cache) { + const contractTxId = entry['contract_id']; + const height = entry['height']; + const state = JSON.parse(entry['state']); + + const blockHeightString = `${height}`.padStart(12, '0'); + const sortKey = `${blockHeightString},${defaultArweaveMs},${sortingLast}`; + + this.logger.debug(`Migrating ${contractTxId} at height ${height}: ${sortKey}`); + + await this.levelDb.put( + { + contractTxId, + sortKey: `${blockHeightString},${defaultArweaveMs},${sortingLast}` + }, + new EvalStateResult(state.state, state.validity, {}) + ); + + result.push({ contractTxId, height, sortKey }); + } + + this.logger.info(`Migration done.`); + + return result; + } +} diff --git a/src/core/Warp.ts b/src/core/Warp.ts index 5520c48..ea33b67 100644 --- a/src/core/Warp.ts +++ b/src/core/Warp.ts @@ -4,7 +4,8 @@ import { HandlerApi, InteractionsLoader, WarpBuilder, - StateEvaluator + StateEvaluator, + EvalStateResult } from '@warp/core'; import Arweave from 'arweave'; import { @@ -16,6 +17,8 @@ import { PstContractImpl } from '@warp/contract'; import { GQLNodeInterface } from '@warp/legacy'; +import { MigrationTool } from '../contract/migration/MigrationTool'; +import { LevelDbCache } from '@warp/cache'; /** * The Warp "motherboard" ;-). @@ -27,19 +30,22 @@ import { GQLNodeInterface } from '@warp/legacy'; */ export class Warp { readonly createContract: CreateContract; + readonly migrationTool: MigrationTool; constructor( readonly arweave: Arweave, + readonly levelDb: LevelDbCache>, readonly definitionLoader: DefinitionLoader, readonly interactionsLoader: InteractionsLoader, readonly executorFactory: ExecutorFactory>, readonly stateEvaluator: StateEvaluator ) { this.createContract = new DefaultCreateContract(arweave); + this.migrationTool = new MigrationTool(arweave, levelDb); } - static builder(arweave: Arweave): WarpBuilder { - return new WarpBuilder(arweave); + static builder(arweave: Arweave, cache: LevelDbCache>): WarpBuilder { + return new WarpBuilder(arweave, cache); } /** diff --git a/src/core/WarpBuilder.ts b/src/core/WarpBuilder.ts index 7501d44..01ea517 100644 --- a/src/core/WarpBuilder.ts +++ b/src/core/WarpBuilder.ts @@ -5,15 +5,17 @@ import { ContractDefinitionLoader, DebuggableExecutorFactory, DefinitionLoader, + EvalStateResult, ExecutorFactory, HandlerApi, InteractionsLoader, + LevelDbCache, MemCache, - WarpGatewayContractDefinitionLoader, - WarpGatewayInteractionsLoader, - Warp, SourceType, - StateEvaluator + StateEvaluator, + Warp, + WarpGatewayContractDefinitionLoader, + WarpGatewayInteractionsLoader } from '@warp'; export const WARP_GW_URL = 'https://d1o5nlqr4okus2.cloudfront.net'; @@ -24,7 +26,7 @@ export class WarpBuilder { private _executorFactory?: ExecutorFactory>; private _stateEvaluator?: StateEvaluator; - constructor(private readonly _arweave: Arweave) {} + constructor(private readonly _arweave: Arweave, private readonly _cache: LevelDbCache>) {} public setDefinitionLoader(value: DefinitionLoader): WarpBuilder { this._definitionLoader = value; @@ -73,6 +75,7 @@ export class WarpBuilder { build(): Warp { return new Warp( this._arweave, + this._cache, this._definitionLoader, this._interactionsLoader, this._executorFactory, diff --git a/src/core/WarpFactory.ts b/src/core/WarpFactory.ts index f8abe5f..6030a8e 100644 --- a/src/core/WarpFactory.ts +++ b/src/core/WarpFactory.ts @@ -55,14 +55,8 @@ export class WarpFactory { * Returns a fully configured {@link Warp} that is using arweave.net compatible gateway * (with a GQL endpoint) for loading the interactions. */ - static arweaveGw( - arweave: Arweave, - cacheOptions: CacheOptions = { - maxStoredTransactions: 20, - inMemory: false - } - ): Warp { - return this.levelDbCached(arweave, cacheOptions).useArweaveGateway().build(); + static arweaveGw(arweave: Arweave, cacheOptions: CacheOptions = defaultCacheOptions): Warp { + return this.custom(arweave, cacheOptions).useArweaveGateway().build(); } /** @@ -71,24 +65,19 @@ export class WarpFactory { static warpGw( arweave: Arweave, gatewayOptions: GatewayOptions = defaultWarpGwOptions, - cacheOptions: CacheOptions = { - maxStoredTransactions: 20, - inMemory: false - } + cacheOptions: CacheOptions = defaultCacheOptions ): Warp { - return this.levelDbCached(arweave, cacheOptions) + return this.custom(arweave, cacheOptions) .useWarpGateway(gatewayOptions.confirmationStatus, gatewayOptions.source, gatewayOptions.address) .build(); } - static levelDbCached(arweave: Arweave, cacheOptions: CacheOptions): WarpBuilder { - const executorFactory = new CacheableExecutorFactory(arweave, new HandlerExecutorFactory(arweave), new MemCache()); - const stateEvaluator = new CacheableStateEvaluator( - arweave, - new LevelDbCache>(cacheOptions), - [new Evolve()] - ); + static custom(arweave: Arweave, cacheOptions: CacheOptions): WarpBuilder { + const cache = new LevelDbCache>(cacheOptions); - return Warp.builder(arweave).setExecutorFactory(executorFactory).setStateEvaluator(stateEvaluator); + const executorFactory = new CacheableExecutorFactory(arweave, new HandlerExecutorFactory(arweave), new MemCache()); + const stateEvaluator = new CacheableStateEvaluator(arweave, cache, [new Evolve()]); + + return Warp.builder(arweave, cache).setExecutorFactory(executorFactory).setStateEvaluator(stateEvaluator); } } diff --git a/src/core/modules/impl/LexicographicalInteractionsSorter.ts b/src/core/modules/impl/LexicographicalInteractionsSorter.ts index df32589..19a107b 100644 --- a/src/core/modules/impl/LexicographicalInteractionsSorter.ts +++ b/src/core/modules/impl/LexicographicalInteractionsSorter.ts @@ -6,7 +6,7 @@ const defaultArweaveMs = ''.padEnd(13, '9'); const defaultArweaveMs_After_Block_973730 = ''.padEnd(13, '0'); export const block_973730 = 973730; -const sortingLast = ''.padEnd(64, 'z'); +export const sortingLast = ''.padEnd(64, 'z'); /** * implementation that is based on current's SDK sorting alg. diff --git a/tools/migrate.ts b/tools/migrate.ts new file mode 100644 index 0000000..222356b --- /dev/null +++ b/tools/migrate.ts @@ -0,0 +1,31 @@ +/* eslint-disable */ +import Arweave from 'arweave'; +import {defaultCacheOptions, defaultWarpGwOptions, LoggerFactory, WarpFactory} from '../src'; + +LoggerFactory.INST.logLevel('debug'); + +async function main() { + const arweave = Arweave.init({ + host: 'arweave.net', // Hostname or IP address for a Arweave host + port: 443, // Port + protocol: 'https', // Network protocol http or https + timeout: 60000, // Network request timeouts in milliseconds + logging: false // Enable network request logging + }); + + const warp = WarpFactory.warpGw(arweave, defaultWarpGwOptions, { + ...defaultCacheOptions, + dbLocation: './tools/.leveldb' + }); + + const result = await warp.migrationTool.migrateSqlite("./tools/sqlite/contracts-3008.sqlite"); + + console.log(result); + + const dump = await warp.stateEvaluator.dumpCache(); + + console.log(dump); +} + + +main().catch((e) => console.error(e)); diff --git a/tools/sqlite/contracts-3008.sqlite b/tools/sqlite/contracts-3008.sqlite new file mode 100644 index 0000000..156956d Binary files /dev/null and b/tools/sqlite/contracts-3008.sqlite differ diff --git a/yarn.lock b/yarn.lock index 1eee243..0e036ca 100644 --- a/yarn.lock +++ b/yarn.lock @@ -7200,7 +7200,7 @@ sprintf-js@~1.0.2: resolved "https://registry.yarnpkg.com/sprintf-js/-/sprintf-js-1.0.3.tgz#04e6926f662895354f3dd015203633b857297e2c" integrity sha512-D9cPgkvLlV3t3IzL0D0YLvGA9Ahk4PcvVwUbN0dSGr1aP0Nrt4AEnTUbuGvquEC0mA64Gqt1fzirlRs5ibXx8g== -sqlite3@^5.0.2, sqlite3@^5.0.3: +sqlite3@^5.0.3: version "5.0.8" resolved "https://registry.yarnpkg.com/sqlite3/-/sqlite3-5.0.8.tgz#b4b7eab7156debec80866ef492e01165b4688272" integrity sha512-f2ACsbSyb2D1qFFcqIXPfFscLtPVOWJr5GmUzYxf4W+0qelu5MWrR+FAQE1d5IUArEltBrzSDxDORG8P/IkqyQ==