Sign Up
Log In
Log In
or
Sign Up
Places
All Projects
Status Monitor
Collapse sidebar
home:marec2000:cryptocoins
stellar
stellar.changes
Overview
Repositories
Revisions
Requests
Users
Attributes
Meta
File stellar.changes of Package stellar
------------------------------------------------------------------- Wed Sep 20 16:50:43 UTC 2023 - marec@detebe.org - Update to version 19.14.0: * Overlay improvements ------------------------------------------------------------------- Thu Aug 03 17:44:30 UTC 2023 - marec@detebe.org - Update to version 19.13.0: * Update soroban env and XDR to take SCError change * !fixup Fix vCurr build error * !fixup format * Support soroban max tx set size upgrade shortcut. * Handle init + init * Conditionalize tracy code. * bump sqlite 3.34.1 -> 3.41.2 * bump asio 1.22.1 -> 1.28.0 * Connect Rust code to Tracy (and upgrade Tracy to latest). * warning police * fix logging issues (compile errors) post switch to fmt 10 * bump spdlog 1.10.0 -> 1.12.0 * bump fmt 8.1.1 -> 10.0.0 * temp entry invariant fix * protocol curr: update meta * Add one more test scenario * Address PR comments * protocol next: update meta * Address comment * Test suite cleanup and audit of source account limit flag * Update lcm * !fixup trying to fix lcm * Compute Soroban write fee based on the bucket list size. * ifdef police * Add tests * Bump env and xdr * Allow configuring initial max tx size for testing * Support max tx size Soroban upgrades * Code move: make upgrade helper functions available for reuse * pull mode test fix * Simplification: remove batchSize completely, default to 100 ops * fix global early catch up timer * check read bytes on every read * Update error * Add test * Error if footprint has expired entry * Comments and improvements * txRate function: remove batchSize parameter, as it only applied to account creation * Make loadgen use unique accounts only for transaction submission * Minor bugfix: loadgen should report num operations for created accounts metrics * Adjust default test flow control value: was too aggressive preventing from submitting larger transactions in loadgen * Remove duplicate declaration and regenerate lcm * Implement rent fees. * Add BucketList Sliding Window and emit size meta * change minor issue in BallotProtocol.h comments * fix * bugfixes, use optional's value for clariry * naming clarity * Remove tag check * Fix tests * cleanup * Validate token contract Asset * Add Footprint LedgerKey validation ------------------------------------------------------------------- Wed Jul 12 12:07:58 UTC 2023 - marec@detebe.org - Update to version 19.12.0: * Update close meta * Switch LedgerTxn instances to read-only in overlay * Disable autobump by default * bump env for wasmi bump * Bump env * Inclusive expirationLedger and use current ledgerSeq for calculation * Added LedgerTxn::restore * Add loadExpiredEntry flag and enforce expiration in mEntryCache * Rebase conflicts * Added lifetime and type info to ConfigUpgrade entries * Fixed SQLite ContractData prefetch bug * Added prefetch expiration tests * Update default network configs for contract costs; bump limits accordingly * Bump env for xdr * Update ledger close meta * Bump xdr and other fixes * Fix tracking in BumpFootprintExpirationOpFrame * Add RestoreFootprintOp * Update ledger close meta * Update env and xdr * Fix more tests * Fix flaky test * Update ledger close meta * make format * bump env * xdr rename * No duplicates in footprint * Use medium thresholds for bump and invoke ops * Bump expiration op * Update ledger close meta * Bump Soroban env & XDR. * Rename tx -> op in Soroban context to avoid confusion, automically enable tx queue limit in vNext * clang-format * Cleanup tx count extraction from a list of resources * test update: mixed limit flag test is not applicable in protocol 20 * tests: make callers explicitly pass all tx set phases * Minor change to Resource functionality to avoid possible collisions * Minor tweak to one of the helper functions in tx set * test: make account generation less confusing * Small test improvement * macro police * Add a test for multiple txs per source account, but introduce a special config to toggle the check; Otherwise, the whole test suite is impacted by the check * Incorporate soroban tx limits into pull mode accounting * Additional tx set validation, but comment out 1 tx/account limit check for now * More tx set tests and other improvements * More tx set tests * Soroban limits: incorporate max tx count limit * Transaction queue: introduce soroban-specific configs and minor cleanup * Resources: ensure we don't overflow in case the limits are ever at max * Soroban tx queue: dedicate a separate multiplier * Make tx size upgrades valid * Update tx meta * Add max tx count for Soroban network settings * Update ledger close meta * Update tx meta * Bump XDR in Core and fix build. * Cleanup functions used in test builds only * Wire up soroban PRNG seed from txset hash, tx and op numbers. * Fix merge conflicts * Re-enable tx meta test * Few more improvements based on review * More improvements based on feedback * Various improvements based on code review * Allow Soroban fee bumps * minor bugfix introduced by the cleanup commit * TxSetFrame tweaks plus new test * Add new tests and fix existing * loadgen hack (temporary) * Temporarily disable tx meta check - meta will need to be regenrated * make a note about pull mode * Add phase support to TxSet and implement Soroban phase * Herder is now using two transaction queues * Helper function to get per-ledger limits * Refactor surge pricing utils to support Resources * Testing: allow specifying network configs for upgrades * Helper function in TransactionFrame to compute per-tx resources * Add new helper class for multi-dimensional limits * Bucket test fix * Fix BucketList EXPIRATION_EXTENSION logic to avoid deleting its own key additions * more renaming lifetime to expiration * enable bumps * sync with env changes * update for env changes * Fix test expectation of an extra diagnostic event -- unclear why * Fix protocol 20 LedgerCloseMeta test vector because bucket content changed * Update dep tree file * update to WIP env change * Fix ifdefs * fixes * Test changes * Fixes for latest env and tests * Commented out broken parts for PR * WIP bump read-only entries * InvokeHostFunctionTests refactor and added LIFETIME_EXTENSION test * Minimum Temp entry lifetime should be larger than autobump * Bug fixes, all unit tests passing * Compiles after rebase * Added ContractData type support to SQL with unit test * Basic InvokeHostFunctionTests working with storage changes * Fixed bug with ContractData types and added unit test * InvokeHostFunctionsTests mostly working * Refactor type helpers and minor XDR updates * Only write LIFETIME_EXTENSION when lifetime changes * WIP bump via footprint * Compiles with XDR rename * Compiles with footprint changes * BucketListDB loads LIFETIME_EXTENSIONS properly * Added BucketListDB lifetime extension tests * Added LIFETIME_EXTENSION merge behavior * Compiles with XDR changes * Update test ledger close meta * Update next tx meta * Update ledger close meta. * !fixup remove debug log * Updates corresponding to the Soroban auth XDR changes. * Simplification * Bugfix: ensure we can always find txs to evict given pre-conditions * Cleanup * Bump env and add xdr mismatch check back in ------------------------------------------------------------------- Wed May 31 15:21:27 UTC 2023 - marec@detebe.org - Update to version 19.11.0: * Avoid BUILD_TESTS code in offline-close, fix other nits * Make sed usage more compat with git refs * Add offline-close command for establishing test network state in quickstart * Reject txs that are too big before placing them into outbound queues * Correct initial network configs for contract cost * Temporarily comment out checkXDRFileIdentity * Add guard * Env bump * Properly update max classic tx size * Bump protocol-next XDR * Fix test * Bugfixes and test improvements * Add missing comment * Fix test so config setting case doesn't loop forever * Update tests with disable functionality and remove obsolete logic * Allow disabling flow control in bytes * Add tests * Add byte limit to flow control * make outbound byte limit a config * Introduce configs to limit flooding in bytes * Update overlay XDR * Update example config * Calculate close time drift based on max slots to remember * Update Github actions to latest * Implement refunds for the Soroban metadata fees. * regenerate Visual C++ files * add missing include (compile error on Windows) * warning police * Update to version of cargo-lock without bug, regenerate dep trees * Cleanup unnecessary AUTH_MSG_FLAG_PULL_MODE_REQUESTED flag * Remove pull version test - not applicable anymore because versions without pull mode are rejected due to min overlay version * Bump overlay min version (to 19.7.0) * Update ledger close meta * Update meta * Bump env and curr dep tree * Bump xdr * Move return values to meta and put the hash of the events and return values in the result instead. * Use flat vector for events now that a soroban tx can only have one op in it * Remove TransactionResultSetFrame and meta hashing * Update tx meta due to txresult removal * remove tx result from meta * Remove code that removed tx result from the archives * Call `maybeComputeSorobanResourceFee` from the fee bump tx frames. * Fix the Soroban resource fee computation bug and add e2e smoke test to cover this. * fix failure * Expose network configs for testing and add tests * Impose contract code and data size limits * Smoke tests for the Soroban fee computation/validation. * - Implement the fee bid (==inclusion fee) computation for Soroban. - Decouple the 'full' transaction fee that includes the 'flat' part (resource fee for Soroban) from the fee bid used for comparisons. The 'full' fee needs to be used in the contexts where we need to reserve the maximum possible fee amount for the account to make sure it can pay the fees. Otherwise, we're mostly interested in fee bids. Some tests semantically probably should use 'full' fee, but since they're not soroban-related, we can use the bid there. With the tx prioritization work we may use a more elegant solution to this. * Fix test for vNext - nomination leader is different because genesis ledger changed * Set initial fee rates and expose bridge fee config structure from the network config. * Expose fee computation function through the rust bridge. * Bump soroban host rev. * Updates based on review * !fixup add an additional metadata limit check. * !fixup Update lcm hashes. * Add the transaction size validation. * Add `Application` to the most of `checkValid` function users. * Plumb tx resources and network config to Soroban host and enforce the relevant apply-time limits. * Add tests for Soroban tx validation. * Transaction & operation frame updates. * Expose soroban network config by reference. * Network configuration fixes for the cost params. * Bump env & xdr * Add tests * Test: ensure nodes can properly vote on values that don't satisfy the limit * Introduce a flag to limit transactions per source account * Add partial initial parameters * Define contract costs network config parameters * Fix missing update to prev-block code * Do more and more-correct soroban version checks in rust code * Fix missing std::endl * Clarify how interface version works in soroban-env * Add missing cases to ConfigUpgradeSetFrame::isValidForApply * Make build steps depend on Makefile not Makefile.am (to handle reconfiguration) * Bump soroban and xdr versions * Fix Soroban lumen transfer test * Support multi-host-fn invocations in `InvokeHostFunctionOp`. * Update ConservationOfLumens invariant to handle the SAC transfers * Add a wrapper for network configuration and load it when appropriate. * Fix versions * add missing ifdef * Further updates, simplifications and enforcement mechanisms * Even further simplification of docs * Simplify versioning plan * Fix small bug in metrics counting contract writes as reads * Remove preflight * Address review comments * Revert part of a3cfa1c35d1, adapt test to CONFIG_SETTING differently * Initial support for contract-host multi-versioning. ------------------------------------------------------------------- Tue May 02 16:25:56 UTC 2023 - marec@detebe.org - Update to version 19.10.0: * Online catchup no longer waits for trigger ledger * Update Visual Studio files * Bump env and xdr * Improve tests: activate flow control across the test suite * Update tests to comply with the new interface * Refactor the flow control logic * Minor cleanup * Revert part of a3cfa1c35d1, adapt test to CONFIG_SETTING differently * sed -> gnu-sed * Update INSTALL.md for MacOS * Bump env * Add various connectivity tests * Refactor: move useful overlay testing functions to OverlayTestUtils * Simulation helper tweak: allow only validators to externalize * Allow adding watchers to Topologies::separate * Minor LoopbackPeer fixes * Bump env * Add check rust vs c++ xdr size check * Update xdr * Add docs * Ignore invalid upgrades instead of throwing an exception * Use debug json for upgrade info * Specify that createUpgradesFor takes the lcl header * Fix flaky test * Update tests * ConfigSettingEntry upgrades * Update the tests to use the new utilities and cover special handling of `CONFIG_SETTING` entries. * Generalize test utils to work around `CONFIG_SETTING` entries. * Prohibit removing `CONFIG_SETTING` ledger entries. * Minor updates to `switch` statements with `CONFIG_SETTING`. * Update `configSettingID` from field to method. * Fix reporting of base XDR git version (was reporting "next") * Visual C++ project and compile fixes * Properly setup rustup in dev container ------------------------------------------------------------------- Wed Apr 05 10:14:14 UTC 2023 - marec@detebe.org - Update to version 19.9.0: * bump env * bump env * update close meta * Update tx meta * Add diagnostic event test * Return diagnostic events in meta on tx failure * Add config setting for diagnostic events * Add support for diagnostic events * Add -D CEREAL_THREAD_SAFE the preprocessor defines * Validators do not save indexes * Better index serialization test * BucketIndex file durable rename * Cleanup * IndexBucketsWork tries to load indexes off disk * Fixed nondeterministic hash bug * Preliminary value overhaul change * Adapt `SnapshotSource` interface * Fixed unit test on gcc * Update test tx meta * Revert "Update expected LedgerCloseMeta file to match changes to contract events" * Separate events by operation * Allow multiple host fn invocations in the same transactions. * Serialize indexes to disk * Allow overlay to work automatically when in loopback mode * use perl instead of sed when generating hashes * fixup packages: make sed and perl dependencies explicit for clarity ------------------------------------------------------------------- Fri Mar 03 16:19:45 UTC 2023 - marec@detebe.org - Update to version 19.8.0: * Add test * Introduce out of sync overlay recovery * Expire messages in outbound queues * Bump env to 0.0.14 * Enable pipefail and check for an error in the perl script so we can fail on stellar-core exceptions * Switch host preflight to recording auth. * bump xdr to match env * Update .dir-locals.el * Bump the env version to latest. * !fixup remove commented out code * Switch back to env main branch. * Fix rust install script * !fixup Handle error * Switch back to the main branch XDR * Formatting fix * Auth next core build * Use auth_next XDR branch * Use local repo for XDR ------------------------------------------------------------------- Sun Feb 05 13:14:52 UTC 2023 - marec@detebe.org - Update to version 19.7.0: * Further restrict nodes with no inbound connections * Limit outbound connections based on inbound config * Support extended survey * docs: fix grammar & spelling in ledger.md * Update the `generateload` command documentation. * Add a new load generation mode that produces DEX transactions. * test the case when the peer doesn't want to turn on pull mode * Clean up outdated logic/metrics/docs/etc * Remove the option to turn off pull mode * Bump the Overlay min version & Overlay version * Bump protocol-next XDR version * Update XDR next submodule * Simplified extended survey * Do not stop survey if there is still backlog * Increase ledger validity buffer in survey * Add missing "override" * Index buckets before adding them to BucketList * bump env * Improvements to survey script * bump env * fix Github actions warning (deprecated features) * Support installing rust on aarch64 * Update example.cfg * Windows build fixes * Herder tests for the lane-related changes. * `TxSetFrame` changes to support multiple lanes. * `TransactionQueue` changes to support multiple lanes. * `TxQueueLimiter` changes to support multiple lanes. * New test utilities to support the new tests. * Change `SurgePricingPriorityQueue` to use lane config instead of a single ops limit. * Introduce configuration parameter to limit maximum number of DEX operations in tx set. * re-run tests that fail under debugger to help diagnose issues faster * Improve compile flags to increase chance of catching issues in CI * Update env version to include delete-im, remove-EnvVal changes * Better bloom filter metrics * Added BucketListDB init tests and invariant checks * survey: give some idea of the work being done * survey: fix crash when augmenting graph * Update VisualStudio files * Refactor some Overlay utility functions * add bucket information in info (verbose only) and catchup output * adjust Visual Studio project to not build vnext files when building vcurrent * update clang-format config * switch to llvm-12 (from llvm-10) and to gcc-10 (from gcc-8) * Remove `FIRST_VERSION_SUPPORTING_FLOW_CONTROL` as the min overlay version is already higher than that. * Remove `mFlowControlState` as all connections must be flow control * Add asserts for flow control accounting. * Set the current ledger age to 0 if it has no LCL ------------------------------------------------------------------- Sun Dec 11 12:16:04 UTC 2022 - marec@detebe.org - Update to version 19.6.0: * Bump soroban env to 0.0.10 release version * Re-record txmeta for protocol-next to adapt to new event structure * Update expected LedgerCloseMeta file to match changes to contract events * Update to new env/xdr/wasmi, expose version info * Windows: warning police * Visual Studio updates * Fix bug in BucketList invariant test * Bump rust toolchain to 1.65. * Migrage libsoci VS project to Postgres 15. * `InvokeHostFunctionOp` changes and corresponding test changes. * Rust changes corresponding to the host changes. * Visual studio project update. * Test & invariant support for contract code ledger entries. * Plumbing for `ContractCode` ledger entries. * Bump Rust deps to use the new XDR/env revs. * Bump XDR revision. * Refactored catchup tests * Revert faulty bucketlist refactor * Fixes * Added metric map to BucketManager * Refactored catchup tests and BucketList * Minor fixes * Additional metrics * Improvements to peer metrics and logging * bump version * Make install script easier to use on macOS * Added metrics * Fixes index race conditions and bloom init * Fix XDRStream::readPage buffer issues and some failbit-handling * ApplyBucketsWork now calls AssumeState, cleanup * Added index garbage collection * Added parallel indexing * Cleaned up config and index interface * Fixed bloom_filter libarary * suggestions * warning/compile fixes (Visual C++) * BucketListDB * Test purging behavior * Test advert caching * Use advert cache * Overlay improvements * Warning police * Improve logging & documentation on connection count adjustment * bump libsodium to use VS2022 * add missing entry libsodium in COPYING * Move to latest version of postgres on Windows * Update wasmi * bump env * Pin `cxxbridge` tool at v 1.0.68 to match `Cargo.lock`. * improve item fetcher tests * expose item fetcher constants ------------------------------------------------------------------- Wed Nov 02 18:20:11 UTC 2022 - marec@detebe.org - Update to version 19.5.0: * adjust demand period in flood test to avoid dropping txs * Bump the min overlay version to 23 that corresponds to 19.3.0 * Build rust code with heavier optimization settings. * clarify timeline for kicking out nodes off the network * bump the overlay protocol version from 24 to 25 as we enabled pull mode by default * Enable pull mode by default (and fix tests that that breaks) * Include `stellar-internal.x` in xdrgen check. * Fix additional linker inputs in VS project. * Vote for accepted values from round leaders * add missing files to VS Project * warning police * remove RustBridge dependency from production code (not used) * Rework Rust dependencies in VS to properly rebuild and link * Bump env and add complex-wasm testcase. * Add some soroban metrics. * Propagate cxx::Exception strings from preflight callbacks * Improve logging * Do not vote for new values after confirming candidates * new metric: referenced values per ledger * Fix nested-ltx bug introduced in #3557 * Include leddger sequence in preflight response * Route results of INVOKE_HOST_FUNCTION to SCVal success * fix VS Project file that was not copying testdata properly * warning police * Make LedgerCloseMeta unit test have deterministic seed. * Fix reinitializeAllGlobalStateWithSeed() to influence NODE_SEED. * Fix typo. * Regenerate meta baseline-next files for v20 * Reformat * Use .json files as test vectors for LedgerCloseMeta test * move feeBump to TxTests.{cpp,h}, consolidate 2 copies of it * Address review comments. * Add a test that checks emitted LedgerCloseMeta looks reasonable. * Update VS project files * Move normalizeMeta to MetaUtils.{cpp,h}, use in TransactionMetaFrame * Add assert that events only occur with 1-op txs * Replace TODOs with issue 3555 references. * Use move semantics more in meta frame classes * fix: properly set result and corresponding hashes in tx meta * fix: properly populate result in LedgerCloseMeta * Regenerate txmeta for vNext * Route events through TransactionMetaV3, TransactionResultSetV2, etc. * Improve event logging and failure-reporting in contract.rs * Add ra-target to .gitignore to allow rust-analyzer a separate space * Rename XDRBuf->CxxBuf, Bytes->RustBuf, for clarity. * Return outputs from Rust in structs; include ContractEvents * Accept a source_account for preflight, route into rust code. * Set LedgerInfo on contract host, fix #3548 * Catch contract host panics, fix #3535 * Remove redundant code * Allow usage of new token host functions * update xdr * Introduce parameters to load generator to skip low fee txs and generate diverse fees. * Replace `generateLoad` arguments with a single config struct. * fixup! pass source account into the host * fix incorrect test that results in a warning * bump-xdr * pass source account into the host * FloodRecord: remove unused filed * Remove unused function * nomination: reuse the value from votes * update installation instruction for macOS * Add methods for identifying DEX-related operations and transactions. ------------------------------------------------------------------- Tue Sep 27 08:17:36 UTC 2022 - marec@detebe.org - Update to version 19.4.0: * Ignore adverts and demands when out of sync * Drop peers that do not respect pull mode settings * Initialize false * enforce line ending to "lf" * fix compile error when building without postgres * Warning police * Properly cleanup the container in `TxQueueLimiter`. * Change INSTALL.md and Dockerfiles to reflect rust installation changes. * Do not install rustc via apt in CI, a newer one is already on the github runner * Rewrite code to be clang-format version agnostic. * Report rust version from "stellar-core version" * Add install-rust.sh * Warning police * Remove redundant flow control reporting * Update ops limit in `TxQueueLimiter` every time we try adding tx. * Ban applied transactions * Add a test that covers the transaction queue eviction logic due to reaching the `TxQueueLimiter` limit. * Made "TransactionQueue limits" tests to rely less on internals of `TxQueueLimiter`. * Test fix: `do not flood too many transactions` had decreased coverage. * Refactor `TxSetFrame` to use `SurgePricingPriorityQueue`. * Refactor `TransactionQueue` to use `SurgePricingPriorityQueue`. * Refactor `TxQueueLimiter` to use `SurgePricingPriorityQueue`. * Use LF line endings for sh files. * Introduce priority queue for performing surge-pricing-related operations. * fix the demand size metric * warning police * Update visual studio files * Pull mode tests * Implement pull mode * Convert hash -> txn using tx queue * TxAdvertQueue stores incoming adverts in a specific order so we can send demands appropriately * Add new XDR structs & fields for pull mode * Add new config values for pull mode * Add new metrics for pull mode * Enable tracy by default for a local docker image * Use soroban-test-wasms crate for test vectors * Update to new xdr-next and sync with rust. * Add VS project file changes to generate XDRFilesSha256.cpp. * Second step of cross-checking XDR file hashes between Rust and C++. * First step of cross-checking XDR file hashes between Rust and C++. * Changed param name 'fee' to 'basefee' in Stellar Documentation * Make it possible to print SCVal using print-xdr * Make tx set frame order matter only during encoding/decoding. * Add preflight and getledgerentry HTTP endpoints for contracts. * Fix rename failures * Bump src/protocol-next/xdr version to pickup persistent state changes * Avoid persisting redundant tx sets * remove 'checkpoint' command from docs as functionality was removed 4 years prior in 7f3e1d07b85def62c5f713d0f99d4d2c3beef5da * Allow passing trusted hash to catchup, warn if catchup is unverified * Test waiting logic in verify checkpoints command * Verify checkpoints command: add ledger buffering check and refactor to allow adding new tests * Catchup manager: add a new helper method * do not force broadcast SCP messages * Add comment explaining regeneration of WASM files * Update contract host to soroban name, recent host drift. * Rename `XDRMatcher.h/cc` to `XDRQuery` to be more general. * Added docs/examples for new features of `dump-ledger` command. * Support for aggregation/group by expressions in `dump-ledger`. * XDR expression support for aggregation statements and field list selection. * Fix InvokeHostFunctionTests * Ignore generated `stack.hh` file. ------------------------------------------------------------------- Fri Aug 05 15:01:02 UTC 2022 - marec@detebe.org - Update to version 19.3.0: * Fix the ordering issue and add the respective check. * make make-mks more resilient * Logging tweaks * Make the casting in MetricTests consistent with the CKMSSample implementation * Medida submodule update (CKMS begin time rounding) * Update the CKMS test to make it less flaky * Add tests for create contract failure cases * rust changes * Update invoke tests to use create_contract host function * Fix wasm examples * Do not use the "define register" hack in VS. * Use --recurse-submodules in make-mks for finding .x files * ifdef-BUILD_TESTS-guard a definition to match its declaration. * Incorporate CAP-42 changes * Populate src/protocol-next/xdr * Remove old attempt at tracking rs-stellar-contract-env as a submodule * Add github.com/stellar-xdr-next as submodule at src/protocol-next/xdr * Remove src/protocol-next/xdr * Decouple tests that record tx-meta from global random engine. * Visual Studio project update. * Support of `GeneralizedTransactionSet` in catchup/simulation. * Support generalized tx set in ledger manager. * Store `GeneralizedTransactionSet` in a separate history table. * Herder changes to support generalized tx sets. * Overlay and SCP driver changes to support `GeneralizedTxSet`. * Refactored `TxSetFrame/TxSetUtils` to support generalized sets and for the general cleanup. * Change `TransactionQueue` to return vector of txs instead of partially built `TxSetFrame`. * Passing-by cleanup: pass tx frames by reference instead of shared ptr in surge pricing comparator. * Support for optional base fee in transaction frame. * XDR changes for CAP-42 and introduce the protocol version 20 supporting it. * Replicate hack for avoiding flex's use of keyword * CAP-0047 - Add envelope types for contractIDs ------------------------------------------------------------------- Sat Jun 25 09:09:03 UTC 2022 - marec@detebe.org - Update to version 19.2.0: * Make `XDRQueryParse` compatible with bison 3.0.4 * Remove `$(srcdir)` from generated files. * Don't return error for `test --list-tests` and similar commands. * Bump overlay version * Fix failing non-test build due to test in `tests/` subdir * Fix contractDataWasm that was broken by https://github.com/stellar/rs-stellar-contract-env/pull/110 * Use head of stellar/rs-stellar-contract-env for stellar-contract-env-host * Change InvokeHostFunctionOpFrame to low threshold * Add conditional compilation to InvokeHostFunctionOpFrame and OperationFrame * Add tests for incorrect parameter number * Add basic tests for InvokeHostFunctionOpFrame * Implement InvokeHostFunctionOpFrame * Implement invoke_host_function * Add machinery to get modified ledger entries back * Allow storage map to be built with deleted entries * Update machinery for storage that maps LedgerKey to LedgerEntry * Remove invoke_contract * Remove ContractUtils and ContractTests * CAP-52 XDR * Fix linker error in loadContractData caused by signature mismatch * Change rs-stellar-contract-host to point to a commit on stellar/rs-stellar-contract-env * Bump Cargo.toml to current main head of rs-stellar-contract-env * Enhance Rust contract-invocation path with storage map and footprint. * Add CAP-0047 and CAP-0053 C++ (CONFIG_SETTING and CONTRACT_DATA). * Add CAP-0047 and CAP-0053 XDR (CONFIG_SETTING and CONTRACT_DATA). * Put fixed limits on contract XDR types. * Remove SCO_BOX. * Switch back from SCV_POS_I64 to SCV_U63 terminology. * Make invariant failures during tests a little easier to debug. * Added utility to dump ledger state from buckets to JSON for debugging (optionally using `jq` for post-processing). * Update tests according to new flow control policy * Example config update and fixes * Remove an option to disable flow control and require peers to have it enabled * Bump minimum overlay version * re-enable ccache per protocol version * Use $(Configuration) variable for the paths of generated files (XDR and Rust). * Added Debug/Release configurations for the current protocol version. * Rename VS build configuration to [Debug/Release]NextV. * Fix broken link in readme.md * Connect Rust's standard log crate to stellar-core's logging system. * Rename src/rust/*.{cpp,h} files to match CamelCase naming style. * Add entries to VS project files * Add initial smart contract system that delegates to rust host. * Rename bridge.{cpp,h} to rust_bridge.{cpp,h} * fix bad dependencies in VS Project files * added missing header ------------------------------------------------------------------- Thu Jun 02 18:18:46 UTC 2022 - marec@detebe.org - Update to version 19.1.0: * Move XDR files to separate subdirs for vCurr and vNext * Minor cleanup * Fixes protocol upgrade bug in BucketManager test * Help ASIO do its autoconfig (as it was broken without) * bump cereal to 1.3.2 * bump spdlog to 1.10.0 * bump Catch to 2.13.9 * bump asio to 1.22.1 * Add a new metric to track transaction latency * fix misses from earlier PR: log direct to stdout for scripts * More and sadder language about upgrade frequencies. * Clarify bucketlist duration arithmetic, again. * Filter out non LEDGER_ENTRYs in captive core * Clarified the required Rust toolchain in Win docs * Fix VS project file * Work around bugs in automake 1.15 on ubuntu 18.04 LTS * Update tests to work with the new TxSetFrame and TxSetUtils * Add back TrimInvalid, clean up removeTxs logic * Check and cache mTxsIsValidHashOrder on TxSetFrame contruction from xdrSet * Refactor on TxSetFrame, no logic change * Refactor TxSetUtils into a static class and clean up * Add RandomEvictionCache for TxSet validity in HerderSCPDriver * Move out utility and previously-mutating methods into TxSetUtils * Remove ABC, repurpose TxSimTxSetFrame and derive it from TxSetFrame * Make TxSetFrame immutable and guarantee txs in hash order * Update .devcontainer/Dockerfile with cargo and rustc * Update Dockerfile.testing with cargo and rustc * Fixup VC++ * warning police * remove default arms in xdr to help clients enforce format * Adjust MSVC project files to build Rust components * preliminary rust/C++ bridge * minor fix: properly handle output file in WriteVerifiedCheckpointHashesWork * Discard an SCPEnvelope with a unsigned StellarValue * Add a test case on discarding a SCPEnvelope with unsigned SCPValues ------------------------------------------------------------------- Tue May 10 18:43:53 UTC 2022 - marec@detebe.org - Update to version 19.0.1: * Filter out non LEDGER_ENTRYs in captive core ------------------------------------------------------------------- Wed Apr 27 07:48:46 UTC 2022 - marec@detebe.org - Update to version 19.0.0: * Document the change in `peers` endpoint * Calculate/expose relevant metrics for `peers` endpoint * Move JSON/metric code in `CommandHandler` to `Peer` * Remove gz files when publish is done * update test tx meta next * update test tx meta current * Bump to protocol 19 * scale ed25519 signers in extraSigners * Update test tx meta current * Update test tx meta next * Add extra signer tests in TxEnvelopeTests * incorporate payload signers into alternative signer tests * Only return LedgerEntry in getDelta * KeyUtils changes for payload signer support * format xdr files * check largest tx seq num in MergeOp * Add CAP-0040 tests * Add CAP-0040 signer * Add xdr for CAP-0040 * Add CAP-0021 tests * Updated tx queue and txset validation * Take v2 ext into account in applyCheck * Take v3 ext into account in test utils * Validate minSeqAge and minSeqLedgerGap * Validate extra signers * Relax seq num check * Validate ledger bounds * Update seqLedger and seqTime * Add ProtocolVersion 19 * Add and use getTimeBounds method * Add XDR for CAP-0021 * Bump overlay version * harden invariant in ballot protocol * avoid possible null dereference when logging * Update txmeta test files for next * Update txmeta test files for current * Change how dest account is selected in loadgen * Always closeLedger in applyCheck * Use TEST_CASE_VERSIONS macro to prevent protocol version rollbacks ------------------------------------------------------------------- Wed Mar 30 16:50:28 UTC 2022 - marec@detebe.org - Update to version 18.5.0: * closeLedger: only call sizeOp once * Improve logging * Minor test adjustments since PEER_READING_CAPACITY does not need to be greater than PEER_FLOOD_READING_CAPACITY anymore * Report connection type when dropping peers * Better flow control defaults * Relax the condition on PEER_READING_CAPACITY as it is safe to do so now * Fix and clarify txSet tests * warning police * bump overlay min=19 (v18.2.0) * Mark last level live entries as init * Switch console logging from stdout to stderr * Write offline-info directly to stdout, bypassing log. * Only log "Config from ..." after log-level is first set. * Fix 'BucketListIsConsistentWithDatabase added entries' * Update docs * Add new flow control metric * Add new idle timeout * Load entry in BucketApplicator when oldest bucket is older than V11 * Stop deleting dead entries for older buckets * Add flow control reporting to peers endpoint * warning police * Update Windows project files to Visual Studio 2022 * bump xdrpp * warning police * delete sqlite file directly when calling "new-db" * Update the quorum endpoint documentation * Update the quorum endpoint logic * Error if ccache enabled but not found * Add brew install ccache to macos instructions * Improve instructions for macOS * Clean up inconsistent usage of LEDGER_PROTOCOL_VERSION * Add TESTING_UPGRADE_LEDGER_PROTOCOL_VERSION to Config * Clean up inconsistent usage of config::CURRENT_LEDGER_PROTOCOL_VERSION ------------------------------------------------------------------- Tue Mar 01 18:12:35 UTC 2022 - marec@detebe.org - Update to version 18.4.0: * bugfix: add proper shutdown methods to Timer and Scheduler * Fix existing tests and add new ones * Document new metrics * Implement flow control * bugfix: ping response logic was wrong * Add additional helper functions in SCP and Herder * Add isFlood helper function * Introduce new configs and XDR for flow control * bugfix: race condition in tests * Allow core to sleep in between tasks for testing * Add @graydon's comment about the metrics * Clear state if stellar-core aborts while applying buckets * Make validators from the same home domain preferred peers * Actually enforce SHA256 contract * Rename maxTxSize to maxTxSetSize for clarity. * Cleanup logic for checking protocol versions. * Rename isString32Valid to isStringValid * Log exception as INFO (instead ERROR) for older protocol version * Adding an option to halt on txINTERNAL_ERROR. * Remove MINIMUM_IDLE_PERCENT and TMP_DIR_PATH from core config. * Add TrustFlagsOpFrameBase to Visual Studio project. * Remove unused method * Return early in updateEntryIfRecorded if entry is not an offer * Update catch2 to 2.13.8 * Add comment about non-ED25519 signers in simulation * Simulate closeTimes more accurately in simulation * Use correct current bucket for future bucket in simulation * Add simulation --verify flag to documentation * Add support for AMMs * Fix fee bump signature scaling * Fix fee bump result check * Add support for missing protocol features * Change SetTrustLineFlagsOp to extend from TrustlineOpFrameBase * Change AllowTrustOp to extend from TrustlineOpFrameBase * Add base class TrustFlagsOpFrameBase to be extended by AllowTrust and SetTrustLineFlag * Default-enable libunwind when capable. * Log as much uncaught-exception info as we can, fixes #3155. * update release notes to include protocol 18 related changes * warning police * bump fmt library to 8.1.1 * bump spdlog to 1.9.2 * Make WorkScheduler executeWork wait on passed in work instead of all children * Only git clean ignored files in Dockerfile.testing ------------------------------------------------------------------- Sun Jan 30 11:43:29 UTC 2022 - marec@detebe.org - Update to version 18.3.0: * fix: close files after applying checkpoint * fix DB rebuild path for captive core * Use HashOfHash in LedgerHashUtils and InternalLedgerEntry * bump medida to pick up latest KMS fixes * Fix transaction flooding test * Fix typo BATCH_SIZE -> BATCHSIZE in CONTRIBUTING.md * Startup: reset DB if in invalid state * Increase retry count when downloading HAS * Update tests and add new ledger buffering test * Artificially delay bucket application for testing * Allow captive core to buffer ledgers on startup * Handle filesystem error gracefully * update install instructions to reference Postgresql * Delete temporary transaction files while catching up. * Add additional offers index * Update test tx meta for new change trust test section * Update BEST_OFFER_DEBUGGING block * Only call getPreviousVersion in getChanges if entry is not Init * Update captive core methods * Remove boolean operator from LedgerEntryPtr * Differentiate between init and live buckets when applying buckets into LedgerTxn * Remove getNewestVersion call from getAllEntries * Use LedgerEntryPtr classes and remove getNewestVersion call from updateEntry * Add LedgerEntryPtr * add missing files to medida's project file (windows) * Add simulation configurations to stellar-core_example.cfg for documentation * Top off trustlines during setup if necessary * Add Visual Studio files for the ConstantProductInvariant * Create large pool and offer for the fuzzer * Add contant product invariant * Allow max int values in fuzzer * Update COPYING to include CKMS * Add tests for CKMSSample * Add HISTOGRAM_WINDOW_SIZE * Update lib/libmedida to include CKMSSample * Do not wait arbitrary amount of time on WorkScheduler shutdown, and ensure callbacks no-op when application is shutting down * Prevent UB if META_DEBUG_LEDGER_SEGMENT_SIZE is 1 ------------------------------------------------------------------- Sun Dec 12 14:04:55 UTC 2021 - marec@detebe.org - Update to version 18.2.0: * txQueue: ban transactions when we drop them late in the pipeline * add base fee information in logs when closing ledgers * bump overlay to 19 and min overlay to 18 * Handle -1 in FLOOD_ARB_TX_BASE_ALLOWANCE config. * Add test for TransactionQueue::findArbitragePaymentLoop * Add arbitrage-tx damping functionality to TransactionQueue * Add TransactionFrameBase::getRawOperations() * Fix off-by-one in BitSet::set. * Factor TarjanSCCCalculator out of QI checker. * Use TRACY_NO_ISA_EXTENSIONS in Makefile.am * Add tests for InMemoryLedgerTxn * Make InMemoryLedgerTxn more strict * Make offer and pool share trustlines searches more efficient in InMemoryLedgerTxn * Clean up MultiOrderBook when offer is removed * Check for DB file existence in captive core * properly convert some enum values when printing json * Enable docker-in-docker inside devcontainer * Update the logic in LogSlowExecution to correctly count the number of dropped messages * Increment iterator before break * Update the API and invariants of mSyncingLedgers to prevent it from growing indefinitely * Fix passive flag xdr comment * enforce FMT_STRING in production code * add a few updates to docs related to meta ingestion with captive-core * Fix libc++ versions in Dockerfile.testing * Provide details of non-checked hashes as fatal messages and throw. * Update visual studio files * Wire --check-test-tx-meta into CI. * Warning police * Add src/test/check-nondet test * Centralize global-state resetting used in test and fuzz. * Clear the signature-verification cache on reseed. * Include protocol version info and prng seed in baselines. * Modify autocheck to use stellar::uniform_int_distribution * Initialize RandHasher explicitly during startup. * Eliminate calls to std::rand, close #2739. * Add and use libc++ std::{uniform_int_distribution,shuffle}, close #2741. * Avoid evaluation-order sensitivity in ManageBuyOfferTests * Add --debug-test-tx-meta option to test.cpp * Switch TxMeta baselines to base64 shorthashes. * Store TxMeta baselines in directory of separate files. * Report 1-line summary at end of TxMeta check runs. * Avoid crashing when TxMeta checks fail. * prefetch entries for offers on revoke * Add entries to mEntryCache ------------------------------------------------------------------- Tue Oct 26 17:06:51 UTC 2021 - marec@detebe.org - Update to version 18.1.0: * fixup: LedgerTxn: avoid looking up the same key in mEntry when we can * InternalLedgerEntry: reduce chance of collision in hash function * fixup: InternalLedgerKey: remove methods that give direct access to internal key * Remove unnecessary protocol version checks in tests * Bump medida yet again * log when meta stream writes are slow * Bump medida version * InternalLedgerKey: remove methods that give direct access to internal key * LedgerTxn: cache hash of InternalLedgerKey * LedgerTxn: avoid looking up the same key in mEntry when we can * LedgerTxn: avoid copying entry when loading an entry in the same LedgerTxn * LedgerTxn: replace iterator for worst offer with a map pattern * LedgerTxn: avoid using "AssetPair" to track offers in multi order book * LedgerTxn: make updateEntry noexcept remove use of multimap * Visual C++ - explicitely ask to preserve frame pointers in optimized builds * LedgerTxn: avoid looking up in "active" when possible * LedgerTxn: use child entry directly to avoid copy * LedgerTxn: modify last modified in place * LedgerTxn: mark as "noexcept" a few functions * Add a new config to wait for merges to complete when catching up * base64 commands: properly parse multi lines * Add ability to only report specific metrics * Remove unused database metrics * Remove unused catchup and app metrics, fix tests * uint128: fix UB * clang-format * Fuzz liquidity pools * uint128: avoid copying uint128s * windows: replace uint128 implementation * avoid including uint128 header when possible * XDROutputFileStream: always use FILE on Windows, cleanup non windows code * fs: add win32 fdopen equivalent * fix: captive core crashes with all invariants enabled * Rename numeric overloads to disambiguate & avoid conversions * Use compiler-native uint128_t when available. * Add uint128_t bench test * Clear TmpDirs before shutting down in the fuzzer ------------------------------------------------------------------- Sun Oct 10 10:02:41 UTC 2021 - marec@detebe.org - Update to version 18.0.3: * temporarily set protocol version to target ledger version when applying buckets * Revert "Make it possible to apply protocol 18 buckets" ------------------------------------------------------------------- Sun Oct 10 10:01:14 UTC 2021 - marec@detebe.org - Update to version 18.0.2: * Make it possible to apply protocol 18 buckets ------------------------------------------------------------------- Tue Oct 05 05:27:51 UTC 2021 - marec@detebe.org - Update to version 18.0.1: * Update isBadPrice to check for 0 amounts * Combine apt-get update and install steps to avoid caching * Ensure docker builds don't use cached layers * warning police * Allow batch processing when using the base64 option for print-xdr and sign-transaction * replace stdin alias used for reading from stdin instead of a file * add support for Asset type in print-xdr * add support for LedgerEntry for print-xdr command * remove obsolete tx flooding code * Add code coverage configure flag ------------------------------------------------------------------- Tue Sep 28 11:05:53 UTC 2021 - marec@detebe.org - Update to version 18.0.0: * Additional deposit/withdraw tests * Revoke multiple pool share trustlines in a revoke sandwich * Test too many subentries when there is a single reserve left * More sponsorship and subEntry checks for pool share trustlines * Update test revoke method to check numSponsoring delta * Update test revoke method to check numSubEntries delta * Add test to redeem pool shares and pull offers in the same revoke * Add test that revokes in between trading transactions * Fix xdr whitespace * Add more pool withdraw with liabilities tests * Test too many sponsoring when there is a single reserve left * Add test to create no claimable balance due to rounding and issuer asset * Add revoke test where assetA is issued by the account getting revoked * Issuer sponsors claimable balances on revoke * Add test to only create one claimable balance on revoke due to rounding * Add test for too many sponsoring on revoke * Update tooManySponsoring to take number of reserves for the first op * address highlighted scenarios from code coverage report * CAP-38: Test 30 bps fee in exchangeWithPool * CAP-38: Test every edge in exchangeWithPool * throw on unreachable cases in deposit * CAP-38: Add safety check that all amounts are positive during deposit * CAP-38: Update tests for bigSqrt round down * CAP-38: Change bigSqrt to round down * Copy assetA before the nested LedgerTxn is created * OfferExchange: fast fail when hitting limits * tests: temporary reduce limits when testing limits * warning police * Remove unnecessary ManageSellOfferOpFrame::isResultSuccess * CAP-38: Update liquidity pool trade tests to test failing on a repeated hop * CAP-38: Update liquidity pool trade tests to swap CUR1 and CUR2 * CAP-38: Add more scenario tests for trading with liquidity pools * Further improvements to filter lambdas * CAP-38: Fix bug with PATH_PAYMENT_STRICT_*_CROSS_SELF * Tests to remove pool share trustlines when below the mininum reserve * fix condition in herder test * fix test: create app at version 0 to avoid downgrade issues * warning police * Add test to reduce pool share trustline limit below balance * Move change trust pool share tests into a separate test case * Add an additional sponsorship offer test * Add test to remove sponsorship from pool share trustline * bump overlay version min/cur to 17/18 * Bump protocol version to 18 * CAP-38: handle receiving 0 in exchangeWithPool, and related hardening * CAP-38: Handle 0 reserves in exchangeWithPool * CAP-38: Add comment to exchangeWithPool explaining why something doesn't overflow * CAP-38: Use addBalance in exchangeWithPool to harden against overflow * CAP-38: Make received-too-much check throw in exchangeWithPool * CAP-38: Fix integer overflow in exchangeWithPool * replace a trustline native check assert with a non soci exception * Update docker image build timestamp format * Additional Herder tests * BucketManagerTests: externalize instead of closing ledger directly * Make ledger trigger function stricter * Allow LedgerManager trigger next ledger * Cleanup ledgerClosed path * CAP-38: Fix "Exchange with liquidity pools" tests * CAP-38: Additional liquidity pool trade tests * Loadgen op count & op apply sleep time config validation * Use addBalance when modifying pool values * Remove log * Improvements to the interface * Bump tracy to take change that disables building with -flto * CAP-0038 - Handle upgrades without the flags field specified * CAP-0038 - handle mismatch due to new flags * CAP-0038 - Update upgrade and info endpoint for ledger flags * CAP-0038 - Add upgrade tests for ledger header flags * CAP-0038 - Check header flags * CAP-0038 - Change isVersionSupported to isOpSupported and pass LedgerHeader * CAP-0038 - Add helpers to check ledger header flags * CAP-0038 - Allow ledger header flag field to be set by upgrades * CAP-0038 - Add xdr for ledger header flags * Make computerMultiplier return int32_t and move test version to TestUtils * CAP-0038 - Allow use of nested LedgerTxns after calling loadPoolShareTrustLinesByAccountAndAsset * CAP-0038 - Change existing OperationIDs to HashIDPreImage * CAP-0038 - Use decrementLiquidityPoolUseCount in ChangeTrustOp * CAP-0038 - Use decrementPoolSharesTrustLineCount in ChangeTrustOp * CAP-0038 - Add initial pool auth revoke tests * CAP-0038 - Move test upgrade helpers to TxTests * CAP-0038 - Update checkLiquidityPool to check poolSharesTrustLineCount * CAP-0038 - Update operations to use removeOffersAndPoolShareTrustLines * CAP-0038 - Add removeOffersAndPoolShareTrustLines to handle pool revoke * CAP-0038 - expose computeMultiplier * CAP-0038 - Move setClaimableBalanceClawbackEnabled into TransactionUtils * CAP-0038 - Get opIndex into AllowTrust and SetTrustLineFlags * CAP-0038 - Add xdr for revoke * warning police * fix compile error (can't use "-" on unsigned) * Add tracy-{gui,capture,csvexport} to bin_PROGRAMS when enabled. * Adjust automatic maintenance to run faster (every 6 minutes) * Add scripts/DiffTracyCSV.py * Bump tracy to take tracy-capture -s change. * Add test for op source acc to clawback sponsored non native balance * Add more CAP-0038 tests * Pass ledgerVersion into SponsorshipTestUtils * Small reordering of protocol 18 behavior in ChangeTrust * LedgerTxn: add capability to "reserve" capacity ahead of time * remove bogus check (size_t is unsigned) * check that tmp folder can get created * Update the tests so autocheck can generate pool share trustlines * Update tests for more coverage on TOO_MANY_SPONSORING and TOO_MANY_SUBENTRIES * Fix tests to handle pool share trustline reserves * Update invariants to validate numSubEntries and numSponsoring sum * Handle pool share trustlines in invariants correctly * Update SponsorshipUtils to handle pool share trustlines * add a new command to gather information on any given bucket * BucketApplicator: commit directly into main ledger when running with "in memory" ledger * Improve ledger hash * LedgerTxn: avoid copying LedgerEntry * Add --enable-tracy-csvexport * Introduce a new mode in LedgerTxn for read-only non-concurrent transactions and utilize it in transaction queue * Move ledger trigger function call out of ledgerClosed ------------------------------------------------------------------- Tue Aug 31 10:05:17 UTC 2021 - marec@detebe.org - Update to version 17.4.0: * Add tests for getPoolID * Preliminary tests trading with liquidity pools * Harden makeChangeTrustAssetPoolShare * Expose convertWithOffersAndPools instead of convertWithOffers * Implement convertWithOffersAndPools * Add XDR for ClaimLiquidityAtom * Use REVOKE_SPONSORSHIP_MALFORMED for all RevokeSponsorshipOpFrame validity failures * Add CONFIGURE_FLAGS argument to Dockerfile.testing * Strengthen upgrade sponsorship tests for invalid keys * Strengthen tests for invalid keys when loading trustlines * Remove unnecessary protocol checks introduced as part of the protocol 15 upgrade process * Windows compile fix * Add tests for hugeDivide * Add tests for exchangeWithPool * Implement exchangeWithPool * Implement huge divide * Bring db-schema.md up to date * Cancel `mSelfCheckTimer` in `ApplicationImpl::gracefulStop` * Work shouldn't be scheduled once WorkScheduler is done * CAP-0038 - make liquidityPoolUseCount int32 instead of int32_t * CAP-0038 - Add visual studio files for LiquidityPoolWithdraw * CAP-0038 - Add initial pool withdraw tests * CAP-0038 - move checkLiquidityPool to TxTests * Record and check TxMeta produced by tests. * Process all SCP messages to resolve NodeID, avoid calling tracking in Herder * Herder state is consistent with catchup * Herder invariance: remove LCL dependency, ensure tracking is ahead or equal to LCL * Herder startup: minor refactor to improve clarity * Update the Visual studio file * CAP-0038 - Add LiquidityPoolWithdrawOp * CAP-0038 - Add xdr for LiquidityPoolWithdrawOp * CAP-0038 - Add method to get pool withdrawal amount * Remove the error message of `sendMessage` based on https://github.com/stellar/stellar-core/issues/2937 * Remove unnecessary throw in ApplyCheckpointWork: invariant failures call abort during operation application * Improve LedgerEntryIsValid invariant for liquidity pools * Add LiquidityPoolDepositOpFrame to Visual Studio project files * Implement LiquidityPoolDepositOpFrame tests * Add testing infrastructure for LiquidityPoolDepositOp * Update ConservationOfLumens invariant for liquidity pools * Add LiquidityPoolDepositOpFrame to Operation::makeHelper * Implement LiquidityPoolDepositOp * Update isAuthorizedToMaintainLiabilities * Add new helper functions to TransactionUtils * Add XDR for LiquidityPoolDeposit * CAP-0038 - make ChangeTrustOp safer by using nested LedgerTxns * CAP-0038 - Add initial pool share asset tests for ChangeTrustOp * CAP-0038 - Make the allowTrust test helper accept alphanum12 assets * CAP-0038 - Update ChangeTrustOp to handle LiquidityPools * CAP-0038 - Do the asset conversion once and store the result * CAP-0038 - Update LedgerEntryIsValid for TrustLineEntryExtensionV2 * CAP-0038 - Add loadLiquidityPool method * CAP-0038 - Add trustlineKey method that takes a TrustLineAsset * CAP-0038 - Add helpers for TrustLineEntryExtensionV2 * CAP-0038 - add xdr for TrustLineEntryExtensionV2 * replace use of "assert" by releaseAssert* * reduce memory usage of asan in ci * Resize ccache in ci-build.sh * Add test for op source acc to claim sponsored non native balance * Simple tests for loadPoolShareTrustLinesByAccountAndAsset * Implement LedgerTxn::Impl::loadPoolShareTrustLinesByAccountAndAsset * Implement LedgerTxn::Impl::getPoolShareTrustLinesByAccountAndAsset * Implement InMemoryLedgerTxnRoot::getPoolShareTrustLinesByAccountAndAsset * Implement LedgerTxnRoot::Impl::getPoolShareTrustLinesByAccountAndAsset * Implement LedgerTxnRoot::Impl::loadPoolShareTrustLinesByAccountAndAsset * Update LedgerTxn interfaces for pool share trust lines * Change primary key of liquiditypool table * Update LedgerTxnLiquidityPoolSQL to expose liquidity pool assets * Bump database schema version * Stop requiring that the elements in OP_APPLY_SLEEP_TIME_WEIGHT_FOR_TESTING add up to 100 for consistency * Implement `pretend` mode and add tests * add documentation and parsing code for `pretend` mode and `LOADGEN_OP_COUNT_FOR_TESTING` ------------------------------------------------------------------- Sun Aug 01 16:18:57 UTC 2021 - marec@detebe.org - Update to version 17.3.0: * Make new CHANGE_TRUST result code negative * Revert "Add a const qualifier" * Revert "Remove redundant loadAccount() from commonValid()" * Revert "Give commonValid() a "fullCheck" parameter" * Revert "Give OperationFrame::checkValid() a third mode" * Revert "Wrap test uses of TransactionFrameBase::checkValid" * Revert "Introduce a "fullCheck" parameter to checkValid()" * Revert "Introduce TransactionFrameBase::getNetworkID" * Revert "Remove FeeBumpTransactionFrame::mNetworkID" * Revert "Test support for new partial form of checkValid()" * Revert "Claimable balance test for partial checkValid" * Revert "Update FeeBump tests for partial checkValid" * Revert "Extend validateTxResults() with partial check" * Revert "Fast-fail transactions quickly determined invalid" * Revert "Weaken params to commonValidPreSourceAccountLoad" * Revert "Replace commonValid()'s two bools with CheckType" * Revert "Add a Peer-level test for fast validation" * remove bogus call to "startOutOfSyncTimer" * warning police (variable shadowing) * Hash LiquidityPoolParameters instead of LiquidityPoolConstantProductParameters * Update switch statements in FuzzerImpl to handle missing cases * retry DNS resolution faster on startup * Add pool share asset test for existing protocols * Add `encode-asset` command for debugging * Revert "Merge pull request #2967 from sisuresh/improve-remove-entry" * CAP-0038 - Update XDRCereal for new asset classes * CAP-0038 - Check for pool share liabilities in invariant * CAP-0038 - Fix simulation * CAP-0038 - Add missing case to assetToString * CAP-0038 - Asset conversions * CAP-0038 - Add and use isIssuer method and templatize getIssuer * CAP-0038 - Update Asset validity helpers * CAP-0038 - Update LedgerHashUtils for TrustLineAsset * CAP-0038 - Add Asset conversion methods * CAP-0038 - Add updated xdr for ChangeTrustOp and TrustLineEntry * CAP-0038 - Remove unused TrustLineWrapper methods * Switch PathPaymentStrictSendTests to ClaimAtom * Switch PathPaymentTests to ClaimAtom * Switch TestMarketOffer::exchanged to produce ClaimAtom * Update transaction subsystem to use ClaimAtom * Add XDR for ClaimAtom * Add simple tests for bigSquareRoot * Implement bigSquareRoot * Add InMemoryLedgerTxn to make in-memory aux table writes transactional. * Fix the loggin logic of LoadGenerator to take care of the division-by-0 case. * Fix bucket counters * Add a Peer-level test for fast validation * Replace commonValid()'s two bools with CheckType * Weaken params to commonValidPreSourceAccountLoad * Fast-fail transactions quickly determined invalid * Extend validateTxResults() with partial check * Update FeeBump tests for partial checkValid * Claimable balance test for partial checkValid * Test support for new partial form of checkValid() * Remove FeeBumpTransactionFrame::mNetworkID * Introduce TransactionFrameBase::getNetworkID * Introduce a "fullCheck" parameter to checkValid() * Wrap test uses of TransactionFrameBase::checkValid * Give OperationFrame::checkValid() a third mode * Give commonValid() a "fullCheck" parameter * Remove redundant loadAccount() from commonValid() * Add a const qualifier * Remove HERDER_TRACKING_LCL_STATE state, use HERDER_TRACKING_NETWORK_STATE in all scenarios for simplicity * Tests: add a helper method to Herder and simplify tests * Tests: always start app, with an option to disable * HerderTests: externalize path * Make state transitions in Herder stricter * Always restore tracking on startup * Deprecate mTrackingSCP and mLastTracking SCP, use Herder::State instead * Message filtering: fix a condition for nodes that have never been in sync * Add visual studio files for LedgerTxnLiquidityPoolSQL * Do schema upgrade for liquidity pools * Fix RevokeSponsorship test and add test for liquidity pool keys * Add liquidityPoolKey helper * Make LedgerEntryIsValid ClaimableBalance checkValid safer * Update tests to handle liquidity pools * Update invariants to handle liquidity pools * Iterate over LedgerEntryType enums * Add ability to compare pool keys and entries * Add ability to hash liquidity pool key * Get pool key from pool entry * Return error if RevokeSponsorshipOp is used with a pool key. Also replace abort with and exception * Update BucketApplicator to handle liquidity pools * Update SponsorshipUtils to handle liquidity pools * Prefetch liquidity pools * Add LedgerTxnLiquidityPoolSQL and related data layer changes * xdr for pool entry * Store LedgerEntry in trustlines table instead of individual fields * Move decode methods to common file * Add unit tests for application setup * Additional startup checks * Minor: Add a name to the Github Workflow * SCP: Add a note about Stellar-types.x in the README * SCP.LocalNode: Use driver method for node's short name * SCP: Break circular dep between SCP and LocalNode * QuorumTracker: Correctly initialize NodeInfo * QuorumChecker: Scope a variable only used inside an 'if' * SCP/QuorumTracker: Use NodeID when referring to nodes, not PublicKey * SignatureUtils: Remove duplicated include * Update removeEntryWithPossibleSponsorship to accept a null owner account when removing a claimable balance ------------------------------------------------------------------- Wed Jun 30 05:27:36 UTC 2021 - marec@detebe.org - Update to version 17.2.0: * Add a BucketListIsConsistentWithDatabase test for filtering * Fix BucketListIsConsistentWithDatabase * Support LedgerCloseMeta on dump-xdr. * Migrate more of util/Fs.{h,cpp} to C++17 <filesystem> * Add meta-debug streaming machinery to LedgerManagerImpl. * Add ledger/FlushAndRotateMetaDebugWork.{cpp,h} * Add METADATA_DEBUG_LEDGERS config var. * Remove potential deadlock when posting from background to main thread. * Probe working-ness of <filesystem> more aggressively. * BucketManager: move garbage collection out of shutdown * Bugfix: process old slots correctly * Add tests for self-check machinery. * Move crypto benchmarking out of tests, add to self-check. * Initialize CommandHandler last * make static arrays safe (old style doesn't enforce proper size at compile time) * Enhance self-check command, add merge-bucketlist command. * Add Application::scheduleSelfCheck * Refactor BucketListIsConsistentWithDatabase, add checkEntireBucketList. * Defer scheduling archive report work, add ledger header check builder. * Add some self-checking helper methods to BucketManager. * Enable reusing VerifyBucketWork by moving some code to DownloadBucketsWork. * Add CheckSingleLedgerHeaderWork. * Remove non-actionable / noisy history archive metrics. * Make GetAndUnzipRemoteFileWork take a retry parameter. * Change ConditionalWork predicate to take Application& * Add support to WorkSequence to continue past first failure. * Add configury for clang and gcc 8 std::filesystem libs * add v17.0.0 to release notes * Change operationID to use an AccountID instead of MuxedAccount * Refactor BucketListGenerator to fix broken acceptance test * fix edge case in "manualClose" * SCP: Remove unused includes of crypto/SHA.h * Only define gExplicitSeed when BUILD_TEST is * remove unused include * Windows: use 64 bit toolchain if available * properly fix warnings signed/unsigned * Remove obsolete test * Simplify reset before applying buckets * Unify rebuild ledger code paths * ApplicationImpl method to upgrade schema and rebuild ledger * Make extension NOT NULL for offers * Remove support for schema 13 upgrade * Bump schema version to force rebuild of offers table * Remove obsolete schema upgrade machinery * Use new rebuild ledger infrastructure for schema 13 upgrade * Use new rebuild ledger infrastructure to initialize ledger tables * Implement maybeRebuildLedger as part of application initialization * Add PersistentState functions for rebuild ledger by type * Add applyBucketsForLCL overload with filter * Add filtering to bucket apply * Remove actBeforeDBSchemaUpgrade * Remove schema upgrade test * scp/SCP.h: Remove import of crypto/SecretKey.h * Minor: Add newline at end of herder/QuorumTracker.cpp * SCPDriver: Make to{ShortString,StrKey} accept NodeID * Remove deprecated --reply-in-memory flag. * Cannot modify namespace std * Guard global with mutex * Test stronger sequence number validation * Update comment in TransactionQueue::toTxSet * Strengthen sequence number validation * Warning police: ODR violating variable in header. * Warning police: multi-line comments. * Warning police: catching polymorphic exceptions by value. * Warning police: signed-unsigned mixing. * Warning police: initialize mMetaStreamWriteTime in right order. * Fuzzer - formatting changes * Fuzzer - Add to documentation * Fuzzer - Add high level overview of the setup * Fuzzer - Add AUTH_IMMUTABLE_FLAG to account * Fuzzer - Add AUTH_CLAWBACK_ENABLED_FLAG to account * Fuzzer - Add more claimable balance variations * Fuzzer - Test limit due to native buying liabilities * Fuzzer - Allow trustline with 0 balance and AUTHORIZED_TO_MAINTAIN_LIABILITIES_FLAG * Fuzzer - Test trustline limits * Fuzzer - simplify state ------------------------------------------------------------------- Sun May 30 17:26:41 UTC 2021 - marec@detebe.org - Update to version 17.1.0: * Restore genesis state when in memory mode * Windows: warning police * implement and use in memory version of resetLedgerState() * refactor code to use resetLedgerState * apply buckets: move reset code to when we reset * do not cancel Workflows in Github Action just let the script fail later - Github actions are not being queued in a reliable order * Small Dockerfile tweaks * Update VS files * Make ApplyBucketsWork independent of merges in progress * add metric to track time spent writing into meta stream * Turn simulate-apply-per-op into configs specifying durations and weights * Ensure stellar-core_example matches defaults * Ensure LM and HM are alive before BM shutdown * Do not try streaming meta in temp apps * Disable meta streaming when creating a temporary app * add missing include file needed when build with clang-10 + GNU-11 * Move docker related files to a new directory * added missing include (thread) * bump xdrpp * Allow rebuilding in-memory state from buckets * Add additional peer loading and storing functionality * Introduce new mode to store ledger headers only * Add official Dockerfile and Makefile * Application initialization: re-arrange bucket and database initilization * Move configuration validation up - check validity right after app creation * Remove all support for signed arguments from uint128_t. * Use string const& in uint128_t * Make uint128_t sign-extension code optional, throw by default. * Disambiguate call to bigDivide in TransactionQueue and tighten up arithmetic. * Add stellar::isRepresentableAsInt64(double) helper. * Reintroduce local modification: UNSAFE_UINT128_OPS macro * Delegate more uniformly to uint128_t methods from template methods. * Fix infinite recursion in non-const template operator|| and operator&& * Update uint128_t from upstream * Add testcases for potential misuse with negative operands. * Correct uint128_t test to cast before constructing * Add scripts/Dockerfile.testing for one-off test images * clang-format 10 * bump gcc/g++ to 8 and llvm to 10 * Mention C++17 in INSTALL.md * Enable in-memory LedgerTxn for in-memory mode * Remove special SQL loading behavior * Reproduce protocol 15 offer corruption * add missing include (Windows) * bump fmt 7.1.3 * sqlite 3.34.1 * bump spdlog to 1.8.5 * Use txbridge in tests to mutate time bounds * Pass TimePoint to TransactionBridge set time methods ------------------------------------------------------------------- Tue Apr 27 11:58:07 UTC 2021 - marec@detebe.org - Update to version 17.0.0: * Add tests for handling timebounds when bridging v0 tx to v1 * fix warnings found with static analysis * pass pedantic flag to xdrc * bump xdr submodule * Update V16 flags to V17 * updated security release notes * fix typos * Minor doc fixup: Link to the referenced file * Tolerate exceptions writing LedgerManagerImpl::mNextMetaToEmit. * Another symbol renaming -- EXPERIMENTAL_PRECAUTION_DELAY_META * bump overlay version min and cur * enable ENABLE_NEXT_PROTOCOL_VERSION_UNSAFE_FOR_PRODUCTION in Windows builds * Use fixed hash for apply order test * Bump protocol version to 17 * Add tests to verify clawback flag isn't cleared incorrectly * Add tests to verify SetTrustlineFlags doesn't incorrectly pull offers * More CAP-0035 tests * Futureproof allow trust validation check * Adapt to recent changes in --in-memory mode in cb386dd * Warning police: call printDistribution on metrics test failure. * Warning police: handle missing TransactionResult codes in throwIf. * Warning police: return-by-move in several cases. * Use std::nullopt in preference to 0-arg std::optional(). * Log root account address when starting new ledger * Adapt to C++17 std::optional * Test UNSAFE_EMIT_META_EARLY configuration * Test !UNSAFE_EMIT_META_EARLY with in-memory replay * Extend live metastream test to test safe meta emit * Provide expected hash in ApplyCheckpointWork * Use expected hash and meta config in ledger close * Add optional expected hash to LedgerCloseData * Introduce and document UNSAFE_EMIT_META_EARLY * Test Herder behavior when processing older externalized slots * Work around some C++17 std::optional fallout in Upgrades.cpp * Do not trigger next ledger if herder is not tracking * Revert "Revert "Merge pull request #2989 from graydon/C17-changes"" * Use random hashes for temp filenames * normalize meta when using printXDR * clarify error message when NODE_HOME_DOMAIN is not set * config: properly quote error messages ------------------------------------------------------------------- Fri Apr 09 15:37:56 UTC 2021 - marec@detebe.org - Update to version 16.0.0: * Enable protocol 16 * Implement logic for protocol 16 upgrade * Use templates to unify loadOffersFromV16 * Make loadOffers much safer * Make decodeOpaqueXDR safer * Prepare loadOffers for protocol version differentiation * Move some features from v16 to v17 ------------------------------------------------------------------- Thu Apr 08 12:05:37 UTC 2021 - marec@detebe.org - Update to version 15.5.0: * ensure that we leave the database in a safe state if force clean is interrupted * clean publish queue when force catchup * add force option to catchup * add methods to delete historical data newer than some ledger number * Minor change * Add global variable identifying the production network * Adjustments to inmemory mode * revert default build to protocol 15 * Revert "Merge pull request #2989 from graydon/C17-changes" * Remove autocheck/apply.hpp, migrate residual bit into generator.hpp * Work around std::shuffle bug in libstdc++-7. * Migrate completely to std::optional for C++17 * Fix some new warnings in C++17 mode. * Move MSVC project files to C++17 * Avoid C++17 MSVC collision with extern __builtin decls * Complete C++17 compatibility fix for soci in MSVC * Remove random_shuffle call that is no longer in C++17 * Changes to get soci to compile under C++17 * Remove get_unexpected call that is no longer in C++17 * Changes that would allow C++17 instead of C++20 * Make a note about AFL_MAP_SIZE * Replace stellar::optional with std::optional * Remove function which is redundant with recent STL * Remove hard-codings of C++14 * Update libcereal to current * logging: removed redundant wrappers * logging: fix logDebug/logTrace * remove port from logs (now in header) * simulation: include peer port in header log line * CAP-0029 - Do not return ALLOW_TRUST_TRUST_NOT_REQUIRED from V16 * Bump protocol version to 16 * Github actions: test both current and next protocol support * ci script: remove "testing toolchain" * github actions: snap to latest version of dependencies * Add missing details to CONTRIBUTING.md * Fix explicit specialization issue in AllowTrustTests * Reuse allow trust tests for set trust line flags * Use flagOp to toggle between operations for second test * Add replacement for for_all_versions * Move all trust tests to the struct * Rename some things to prepare to transition the other test case * Reuse authorized to maintain liabilities tests for set trust line flags * Use flagOp to toggle between operations * Add helper functions to replace for_versions * Change authorized to maintain liabilities struct to a template and add some helpers * Create a struct for authorized to maintain liabilities test * Use order-book-not-crossed invariant when fuzzing * Windows build of order-book-not-crossed invariant * Unit tests for order-book-not-crossed invariant * Order-book-not-crossed invariant implementation * Call new snapshot/reset interfaces from fuzzer * Add {snapshot,reset}forFuzzer to InvariantManager * Build resetForFuzzer() under BUILD_TESTS * Move generateOffer to InvariantTestUtils to export * Captive core uses any available archive to catchup * Make apt mirror configurable * Add missing include * Improve auth revoke offer test * SetTrustLineFlags - Update Visual Studio files * SetTrustLineFlags - Add invariant check to make sure trustline clawback flag is never enabled on an existing trustline * SetTrustLineFlags - Modify existing offer and payment tests to also use SetTrustLineFlags op * SetTrustLineFlags - Add tests for SetTrustLineFlags op * SetTrustLineFlags - Add plumbing to test SetTrustLineFlags op * SetTrustLineFlags - Add SetTrustLineFlagsOpFrame * SetTrustLineFlags - Add isAuthorizedToMaintainLiabilities method with a flag parameter * SetTrustLineFlags - Split up mask check from auth combo check * SetTrustLineFlags - Use TRUSTLINE_AUTH_FLAGS * SetTrustLineFlags - Use child LedgerTxn in removeOffersByAccountAndAsset * SetTrustLineFlags - Use removeOffersByAccountAndAsset method * SetTrustLineFlags - Add xdr * Add test for LedgerTxnRoot best offers cache eviction ------------------------------------------------------------------- Tue Mar 30 16:40:23 UTC 2021 - marec@detebe.org - Update to version 15.4.0: * fix apt mirroring in dev container, now requires CAs * Add src/archtmp-* to gitignore * Various improvements to survey * libmedida defaults to "sliding" for all metrics * pickup version of medida that uses sliding window by default for everything * Reject peer early if no pending slots are available * Check for memcpy() of null in signHashX * Correct order of offer parameters in fuzzer setup * Relax assertion to allow empty fuzzer setup config * Add *.o.tmp to gitignore * remove mention of Ubuntu 16 * Update dev container to use focal instead of xenial * Implement best-offer debugging * Pass best-offer debugging Config to LedgerTxn * Set best-offer debugging Config option in tests * Run-time Config option for best-offer debugging * Compile-time flag for best-offer debugging * Delete everything related to `InferredQuorum.{cpp,h}` and `InferredQuorumUtils.{cpp,h}` * Improve naming of externalize metrics * makeValid for claimable balance must produce v0 before protocol 16 * fix: fix the enum-body format in XDR. * Make CreateClaimableBalance free functions static * CB Clawback - Update Visual Studio files * CB Clawback - Update LedgerEntryIsValid * CB Clawback - Add tests * CB Clawback - Set clawback flag on claimable balances * CB Clawback - Add ClawbackClaimableBalanceOp * Factor out reducing trust line balances/limits * Factor out reduceNativeBalancesAfterSetup() * Factor out TransactionFuzzer::initializeOffers() * Factor out initializeClaimableBalances() * Factor initializeTrustLines() out of fuzzer setup * Factor initializeAccounts() out of fuzzer setup * Reduce trust line limits after fuzzer setup * Factor TrustLineParameters out of AccountParameters * Allow fuzzer setup offers to be sponsored * Refactor ClaimableBalanceParameters * Define emplaceConditionallySponsored() * Add SponsoredEntryParameters * Make AssetID more flexible * Relocate the definition of AssetID * Increase the range of fuzz-generated account IDs * Make fuzzer asset decoding more flexible * Add an ordering test * Avoid overflow in ConservationOfLumensTests * Avoid overflow in ExchangeTests with bigMultiply() * Fix overflow in BumpSequence "large bump" test * When RUN_STANDALONE = true, archives should be optional * CB Clawback - Add isClawbackEnabled to TrustLineWrapper * CB Clawback - Add xdr * Add tests to issue/burn more than INT64_MAX in a single path payment * Allow strict order of transaction execution for some tests * Update tests to comply with the StellarValue and close time updates * run automatic maintenance more often with smaller counts * Sort stored ledger keys after fuzzer setup * Reseed autocheck before fuzzer setup * Sweep the codebase for STELLAR_VALUE_BASIC, remove unneeded close time checks * bump overlay version to 16 * warning police * Make xdr_to_string use xdr::archive * Make the name field of xdr_to_string mandatory since otherwise Cereal would print "value0" * Update the way a container is printed * Add issuer information when printing Asset * Strengthen assertion in applySetupOperations * Mark fuzz transaction failed if validation fails * Pass consistent parameters to applySetupOperations * Improve a local variable name * Use true randomness in genFuzz() * Map ClaimableBalanceID and LedgerKey to one byte * Break up attemptToApplyOps() into two functions * Make fuzzer XDR compactor/unpacker aware of Asset * Document fuzz performance testing in fuzzing.md * Create a sponsorship during fuzzer setup * Create claimable balances during fuzzer setup * Process seqnums when applying fuzzer transactions * Make a PublicKey parameter into a const& * Fuzzer logging enhancements * Add --disable-libunwind support to configure.ac * Fix end-transaction iterator computation in fuzzer * Add fuzzer setup parameter for passive offers * Modify tests for merge op bug fix * Move a couple locals closer to their use * Move asset issuance policy into AccountParameters * Introduce fuzzing account parameters structure * Bug fix - allow merge into sponsoring account * Improve SQL pool connection set-up ------------------------------------------------------------------- Tue Feb 23 08:55:48 UTC 2021 - marec@detebe.org - Update to version 15.3.0: * Handle failures when setting logging to file * overlay:only mark broadcast metric if a message was actually sent * Separate fuzzer setup parameters from operations * Add warning if entry cache size is too small * Prefetch for offers that have already been loaded * Rename mBestOffersCache to mBestOffers * Remove unused best offers cache size config value * Change mBestOffersCache to use an unordered_map * Remove prefetching limit * txqueue limiter: clarify logic in "canAddTx" * txqueue: warn when queue is configured to reject all txs above a certain size * extract surge pricing related helpers to their own files * tx queue: return a fee hint when rejecting transactions in limit code * txqueue: give better name and types to tx queue parameters * tx queue: switch default to delayed flooding * Config: change FLOOD_OP_RATE_PER_LEDGER to double and default to 1.0 * txqueue: broadcast transactions in surge pricing order * tx queue: extract lessThanXored * math: add non const rand_element variant * flood tests: test both variations (async and sync broadcast) * txqueue: reduce multiplier for how many transactions we keep in memory * flood tests: fix as to not close ledgers while performing flood tests * tx queue: require a minimum base fee after evicting transactions * tx queue: change limit policy to use surge pricing order * tx set: extract surge pricing code * txqueue: refactor limit handling * txqueue: cleanup tests * warning police * logging:do not use reserved names (Windows) * switch to gcc-7 * update build instructions to use clang-8 to build in CI * use clang-format-8.0.0 * format with clang-format-8 * Do not enforce bucket oldest version check * warnings: remove unused vars * fix disable-spdlog * Add change trust and allow trust V16 validity checks for operating on self * fix wrong comment for CreatePassiveSellOfferOp * Clawback - Change AssetCode parameter to Asset * Add the `flatten` command to OverlaySurvey.py * Improve HAS validation * Use root account consistently in fuzzer setup * Clawback - Update Visual Studio files * Clawback - Update LedgerEntryIsValid and tests * Clawback - Add tests * Clawback - Move setAuthorized to AllowTrust * Clawback - Update AllowTrust * Clawback - Add ClawbackOp * Clawback - getAsset - throw on unexpected assetCode type * Clawback - Use common method to create Asset * Clawback - Add addBalanceSkipAuthorization method * Clawback - Add CLAWBACK_ENABLED flag to trustline * Clawback - Add AUTH_CLAWBACK_ENABLED_FLAG * Clawback - Add xdr * Ran clang-format * Address review comments. * Use iterator interface for attemptToApplyOps() * Reduce account balances after fuzzing setup * Clean up TransactionFuzzer::inject() * Make attemptToApplyOps() limit transaction size * Refactor, rename and simplify process shutdown tracking machinery (NFC). * Simplify and make more-robust ~ProcessManagerImpl and shutdown. * Add tryProcessShutdownAll helper * Extract ProcessManagerImpl::reapChildren from bottom of handleSignalChild. * Rename SIGCHLD-handling functions. * Add getNumRunningOrShuttingDownProcesses to include not-yet-reaped count. * Add comments explaining existing ProcessManager interface contracts. * Fix error routing on windows. * Report exceptions during shutdown a little more clearly. * bump asio to 1.18.1 * Generate fuzz data non-deterministically * Ignore fuzz-findings and min-testcases anywhere * Avoid generating DEADENTRYs that interfere with test. Fixes #2874. * Improve determinism in shorthash when testing. * Improve determinism in ledger entry generation. ------------------------------------------------------------------- Wed Jan 20 06:38:20 UTC 2021 - marec@detebe.org - Update to version 15.2.0: * Remove getDelta call after operations are applied * Truncate log files manually rather than via spdlog. Fixes #2867. * Only truncate log on logrotate, not startup. Fixes #2868. * Make waiting mechanism a property of BasicWork * use toStellarMessage when possible * tx queue: smooth out amount of operations per flood period * txqueue: work around a libstdc++ bug * tx queue: add delayed broadcasting test * tx queue: perform a linear interpolation for determining the number of operations to flood * tx queue: keep track of number of ops to broadcast in AccountState * tx queue: refactor broadcast code * tx queue: hook up delayed flooding to config * tx queue: add (naive) delayed broadcasting * tx queue: keep track of which transactions got broadcasted * tx queue: re-implement randomized broadcast * tx queue: properly account for transactions getting broadcasted * overlay: broadcastMessage returns a hint that it actually did something * herder: make it the responsibility of the tx queue to broadcast * tx queue: simplify obsolete upgrade code (protocol 13) * herder: add some additional information when doing some out of sync work * Use sort -R instead of shuf in run-selftest-pg * Don't generate operation meta if an operation has already failed * Make LedgerTxnEntry bool operator stricter * Update ci-build.sh to enable next protocol version * Add configuration flag to enable next protocol version * RunCommandWork: prevent callback from firing after work is done * Log content hash on txINTERNAL_ERROR * Fix typo in LedgerTxn::getImpl error message * fix ascii art in LedgerTxn.h * Switch build status icon to GA * Tell libsodium not to download code from savannah.gnu.org in autogen.sh. * fix crash when changing a partition log level with the wrong case * fix a format typo in new logging * warning police and cleanup * add missing file to vs studio project file * dev container: do not use sudo as it's not needed, do not purge * dev container: use mirrors to speed up downloads * dev container: improve sync instructions * Remove Easylogging++, default-enable spdlog. * Restore Herder state during in memory replay * Add support for spdlog logging, switch to fmt-based CLOG calls. * Create initial fuzzer accounts by using operations * Escape shell variable in Makefile.am * Strengthen transaction set sequence number validation condition * PR feedback * fuzzer: use compact binary contract with fuzzer * fuzzer: refactor how pregen accounts are * fuzzer: always add a source account to operations by default * fuzzer: don't wrap operations in a stream * fix fuzzer logging * revive fuzzing code * cancel stale GA workflows when PRs/branches get updated * Remove unused parameter from SignatureChecker * fix formatting in overlay * Add the self-check command * Run self-check on startup * Add self-check http endpoint * Update stellar-core.vcxproj * Add HistoryArchiveReportWork for history related sanity checkes * Cleanup type update in PeerManager * Explicitly select RNG type for gRandomEngine * Disable backtraces on non-darwin clang. libunwind crashes. * Disable backtraces on non-main thread. * Add backtrace-printing to std::terminate. * Add STELLAR_NO_BACKTRACE emergency-disable env var for backtraces. * Update VS project files to include Backtrace * Add backtrace-printing machinery and use it in GlobalChecks asserts. * mark build as failed if there is nothing to do * fix build script when ccache folder is missing * Use the correct status for 404 responses * Fix close time cutoff when it is too small * catchup tests: rename misleading fields * tests: compute timeout based on amount of work in catchup test * refresh security release notes * Set MAX_BEST_OFFERS_BATCH_SIZE to PREFETCH_BATCH_SIZE * Remove the <= and bool operators from the BitSet class * fix mergeKey stringifier * use UnordereredMap instead of std::unordered_map * use UnorderedSet instead of std::unordered_set * add RandHasher and UnorderedMap/Set helper classes * rename travis-build.sh into the more generic ci-build.sh * build script: keep default behavior for the compiler being used * reduce maximum size of cache * purge cache when too old * work around stale caching issues in Github actions ------------------------------------------------------------------- Tue Nov 24 16:55:37 UTC 2020 - marec@detebe.org - Update to version 15.1.0: * bump overlay min version to 15 * properly sign transactions in tests * fix flaky herder test * Add tests for operation filter configuration * Update VS project files to remove LoadManager and tests. * Remove LoadManagerTests. * Remove VirtualClock idle-crank estimation machinery, dead code. * Remove DB idle-time and timer-excluder machinery, all dead code. * Remove LoadManager, has not worked in a while. * update Visual Studio project files * Cache the results of isAQuorum * Add maybeGet to RandomEvictionCache * Minor optimization of contractToMaximalQuorum * enable GH actions on auto branch * Update metrics.md with overlay.message.drop * Drop messages in the write queues as well when overloaded. * Update VS project files to include BLAKE2. * Switch some hot, private strong hashes from SHA256 to BLAKE2. * Change a bunch of sha256(xdr_to_opaque(...)) calls to xdrSha256(...) * Add BLAKE2 wrapper. * Move CryptoError to its own header. * Fix broadcast test. * Split broadcasts into multiple Scheduler Actions. * Make a dynamic assert static. * Shave some costs in Logging::log{Debug,Trace}, use in TCPPeer::startRead * Make Peer::recvMessage build fewer strings. * Make Peer::toString avoid making new strings. * Make PeerBareAddress avoid rebuilding strings per toString call * add github action to run tests on PRs * travis.sh : separate g++ from clang++ * travis script: add use-temp-db option * Add more revoke sponsorship tests * Create and use helper for invalid assets * Update VS project files. * Limit LogSlowExecution to one message per second (plus overflow summary). * Added support for filtering transactions by operation in Herder * Simplify status reporting in testtx endpoint * bump asio to 1.18.0 * bump sqlite to 3.33.0 * bump catch to 2.13.3 * warning police * bump fmt to 7.0.3 * Rename GeneralizedLedgerEntry to InternalLedgerEntry and GeneralizedLedgerKey to InternalLedgerKey * Renamed GeneralizedLedgerEntry.{h,cpp} to InternalLedgerEntry.{h,cpp} and update includes * Report possibly problematic archives * Split async-read issues into per-peer queues, to balance input pressure. * DEBUG-guard a log message on the receive path. * Fix loadgen parameter parsing * Herder: move responsibility of reporting outliers to PendingEnvelopes * herder: move cost reporting to PendingEnvelopes * math: fix implementation of k-means++ * Herder: add new tests for slot retention policy * herder: extract `eraseBelow` as to share it from call sites * PendingEnvelope: continue to download active slots * PendingEnvelope: rename `slotClosed` to simply `forceRebuildQuorum` * PendingEnvelope: centralize cleanup code into `eraseBelow` * PendingEnvelope: separate `stopAllBelow` functionality from `slotClosed` * Overlay: change "ledgerClosed" into "clearLedgersBelow" * Herder: move cleanup code into "LedgerClosed" helper function * Herder: add new tests for slot retention policy * Herder: improve slot purging policy when out of sync * herder: move OUT_OF_SYNC_RECOVERY_TIMER * SCP: code cleanup * SCP: add "v-blocking" property to slot * SCP: add processSlotsDescendingFrom * herder: limit number of SCP slots we forward * Herder: move signature verification after fast-fail checks * bump min overlay version to 14 * fix typo in peer's logging code * Update maybeParseParam and paraseParam * Bug #2722 - Remove MAX_BATCH_READ_{COUNT,PERIOD_MS}, obsoleted by VirtualClock::shouldYield * Replace maxCommit() = (N / 2) + 1 with N / 2 for possibly faster quorum intersection checks * add stale content hash detector ------------------------------------------------------------------- Tue Oct 27 21:00:16 UTC 2020 - marec@detebe.org - Update to version 15.0.0: * bump overlay version to 15 * check for invalid asset in trustline data layer * tests: create databases with a random UTF encoding if possible * Revert "temporarily disable some tests" * temporarily disable some tests * Produce backtrace in Travis * additional revoke sponsorship test * Add special exception for data layer along with test cases * Add revoke sponsorship input validity tests * Add claimable balance test where source account is issuer * Add revoke sponsorship tests for new validity checks * Move too many sponsoring tests to avoid nested for_versions * Do revoke sponsorship tests in all versions from 14 * Fix validity checks for RevokeSponsorshipOpFrame * Emphasize negative offer id tets for offer operations * Do not allow negative offerID in offer operations * Bump to protocol version 15 * Do not use config for genesis in claimable balance tests * Pretty-printing xstring using Cereal * Change a bunch of asserts to releaseAsserts. * Add and use futureIsReady helper, fixing bug #2757. * Fix test to exercise inter-work future propagation, catching bug #2757. * SCP - Nomination: fast fail rounds that would not pick a new round leader * SCP nomination stats: properly reset slot between simulation rounds * warning/compile police * overlay: fix test that would crank forever * overlay: always enable "ping" * herder: increase the timeout for the out of sync recovery process * herder: replace rebroadcast timer by an out of sync timer * Rename uso which stands for UpdateSponsorshipOp to rso which stands for RevokeSponsorshipOp * Replace ConfirmAndClearSponsor with EndSponsoringFutureReserves * Replace SponsorFutureReserves with BeginSponsoringFutureReserves * Replace UpdateSponsorship with RevokeSponsorship * Update includes for new file names * Rename files associated with sponsorship operations * properly quote test spec for make check * Update VS project files * fix windows project file * Cost reporting test in an assymetric topology * Herder: add cost reporting for slots * Implement k-means clustering algorithm and add tests * PendingEnvelopes: record cost of envelopes * Test new quorum tracker * QuorumTracker: record distances to the local nodes * Improve utility function and allow fast failing * Use the special name (stdin) for conf * Use the special name (stdin) for print-xdr * Use the special name (stdin) for sign-transaction * Create a special name for stdin * overlay: properly connect to preferred peers * Replace xdr::xdr_to_string with xdr_to_string * Remove xdr_printer and use xdr_to_string instead * Use xdr_to_string in printOneXdr * Add xdr_to_string that uses Cereal to XDRCereal.h * Move non-templated cereal_override to XDRCereal.cpp * Add the license notice to XDRCereal.h * SCPDriver: Make hashing routine user-definable * overlay: avoid reallocating vectors * herder: never ask for SCP messages that we'd be dropping * herder: use a smaller limit when asking for older SCP messages * herder: share computation of which ledger to ask peers when requesting more SCP messages * isQuorumSetSane: Add error string parameter * Take stellar-core test parameters from environment * Add a new overload of closeLedgerOn() * Replace some +(char*, string) with fmt::format() * Correct a class name in a comment * Ignore a couple of test output directories * Fix flaky liabilities test * Update VisualStudio project files for sponsorship invariant * Add tests for SponsorshipCountIsValid invariant * Fix makeValid for ClaimableBalance * Update LedgerEntryIsValid to check signerSponsoringIDs * Add SponsorshipCountIsValid invariant * Comment update * bump xdrpp for minor spec-compliance fix * Improve connection management flow ------------------------------------------------------------------- Fri Oct 02 16:19:23 UTC 2020 - marec@detebe.org - Update to version 14.1.1: * fix tests with bad signatures * give a sensible default value for operations error code ------------------------------------------------------------------- Tue Sep 29 13:06:42 UTC 2020 - marec@detebe.org - Update to version 14.1.0: * Reseed autocheck::rng() along with other RNGs * properly handle "failbit" in streams * Do not special case empty buckets * warning police * Add a sanity check on the networkPassphrase * Add networkPassphrase to HAS * Add networkPassphrase to test files * Typo fixes * Add load to the Serialization round trip test * Print names for enum values * Move cereal_override's to its own header * Remove the json flag and add the compact flag * Make printOneXdr use cereal instead of xdr_to_string * Record externalize lag for nodes in qset * Restrict disabling the Herder's rebroadcast timer * Correct Herder::MAX_TIME_SLIP_SECONDS link error * Tests for enhanced "manualclose" * Implement enhanced manual close * Allow to set HTTP port in config to 0 as a way to disable it * warning police * SCP: remove dependency from Config.h * Ballot protocol: simplify `validateValues` * Docs: correct defaults * Bump tracy to get sample_max_stack kernel version check * Update docs * Wire up catchup-before-run arguments. * Add no-op args for --start-at-{ledger,hash} * Add --in-memory support to "run" command. * Add --in-memory option, deprecate --replay-in-memory. * Fix trustline invariant message * Refactor LedgerTxnRoot::getBestOffers to share code * Un-inline Bucket::checkProtocolLegality() * Add testcases for LedgerEntryIsValid invariant * Remove unused method * Add check in LedgerEntryIsValid to ensure claimable balances are immutable * Avoid spec-violating memcpy with nullptr in 0-byte case. * Make --disable-easylogging build on linux. * Add comments and clarify finalization of incremental SipHash. * fix balance issue in generateBuyingLiabilities * remove incorrect uses of getLastMinBalance * warning police * Windows compile fix * fix Visual Studio project file to use USE_EASYLOGGING * Add configuration flag to disable easylogging * Print hashes with binToHex instead of as opaque arrays * Include-what-you-use before modifying util/Logging.h * make quorum tracker use only what it needs * Replace an unordered_set in SurveyManager with a queue * Add sponsorship signer test * Add UpdateSponsorship error tests * Add and use hasAccountEntryExtV2 function * Add additional sponsorship tests for SetOptions * Update cereal to v1.3.0 * allow access to HTTP endpoint -- /info in particular -- during catchup. * Bump tracy submodule and build with -DTRACY_ONLY_IPV4 * Add comment explaining arithmetic in fee bid * add missing test when merging sponsored accounts * scale sponsorships in simulation * Add claimable balance support to simulation * Remove residual TODOs * Allow AccountEntryExtensionV2 in tests * Update protocol release notes for v13.0.0 and v14.0.0 * Add test for sponsored one-time signer on fee bump * Add test to pull sponsored offers when trustline auth is revoked * Herder tests: test trigger ledger when back in sync * squash database schema version 12 * properly implement Windows version of sign-transaction * squash database schema version 11 * squash database schema version 10 * Add sponsorship tests for path payment * check offer entry sponsorship in tests * add opTOO_MANY_SPONSORING tests * Herder: trigger next ledger if Herder detected LM getting back in sync * Tests: reflect LM getting back in sync changes * LedgerManager: get back in sync as soon as catchup is done ------------------------------------------------------------------- Fri Aug 21 13:34:06 UTC 2020 - marec@detebe.org - Update to version 14.0.0: * Check correct metric * increase timeout for test "connecting to saturated nodes" * improve how extrachecks are enabled * Fix sponsorship bug in MergeOp and add tests * Fix warning in transactionFrameFromOps * Remove duplication in CAP-0033 tests * Add missing include (compile error) * warning police * Windows: fix visual studio project file * TEMPORARY COMMIT: Avoid extension v2 in randomly generated accounts * Prevent side-effects leaking from operations in protocol version 14 * Move normalizeSigners to InvariantTestUtils and account for signerSponsoringIDs * Remove unused getMinBalance method * Update liabilities tests for sponsorships * Remove addNumEntries * Update VisualStudio files * Refactor repeated find-signers-by-key into findSignersByKey in TransactionUtils * Fix schema upgrade test for v2 extensions * Add sponsored one time signer test * Update ClaimableBalanceTests for sponsorships * Update OfferTests for sponsorship * Update SetOptionsTests for sponsorship * Update ManageDataTests for sponsorship * Update ChangeTrustTests for sponsorship * Update CreateAccountTests for sponsorship * Stop using USE_CONFIG_FOR_GENESIS in CreateAccount tests * Refactor CreateAccount tests out of PaymentTests * Add tests for UpdateSponsorship * Add tests for ConfirmAndClearSponsor * Add tests for SponsorFutureReserves * Introduce SponsorshipTestUtils to facilitate testing * Add new operations to TxTests * Return public key as a const reference in TestAccount * Take care of sponsorships on upgrades * Add invariant for ClaimableBalances to always check for sponsorship * Remove references to createdBy/reserve for ClaimableBalance and use sponsorship mechanism * ClaimableBalance XDR change to use sponsorships * Update SetOptionsOpFrame for sponsorships * Further simplifications to SetOptionsOpFrame::addOrChangeSigner and SetOptionsOpFrame::deleteSigner * Refactor SetOptionsOpFrame * Update TransactionFrame and FeeBumpTransactionFrame for sponsorship * Update MergeOpFrame for sponsorship * Update ManageOfferOpFrameBase and OfferExchange for sponsorship * Update AllowTrustOpFrame for sponsorship * Update ManageDataOpFrame for sponsorship * Update ChangeTrustOpFrame for sponsorship * Update CreateAccountOpFrame for sponsorship * Implement UpdateSponsorshipOpFrame * Implement ConfirmAndClearSponsorOpFrame * Implement SponsorFutureReservesOpFrame * Check for residual sponsorship * Introduce SponsorshipUtils * Add prepare extension functions to TransactionUtils * Update interface of getMinBalance in TransactionUtils to facilitate sponsorships * Update getMinBalance for sponsorships * Refactor AccountEntry and TrustLineEntry extension V1 initialization * Add functions to create and load SponsorshipKey and SponsorshipCounterKey * Update XDR for CAP-0033 * Update GeneralizedLedgerEntry, GeneralizedLedgerKey, and LedgerHashUtils for sponsorship counters * Introduce SponsorshipCounterKey and SponsorshipCounterEntry * Update GeneralizedLedgerEntry, GeneralizedLedgerKey, and LedgerHashUtils for sponsorships * Introduce SponsorshipKey and SponsorshipEntry * Test LedgerTxn with generalized ledger entries * Update LedgerTxn for GeneralizedLedgerEntry and GeneralizedLedgerKey * Introduce GeneralizedLedgerEntry and GeneralizedLedgerKey * Remove redundant AFTER predicates * herder: ensure that we're downloading what we're missing * overlay: make trackers idempotent * Add NOT predicate for Claimable Balances * replace all remaining LRUcache by RandomEvictionCache * fix crash on shutdown (as to get the proper error message) * Fix copy/paste bug in ledger transaction prefetch * herder: don't reject messages as aggressively during bootstrap * overlay: force rebroadcast actually rebroadcasts * Add a short explanation of what Custom-A is * add missing signature in flooding test * Adjust close time * Allow to manually close ledger when waiting for consensus * bump overlay cur=14 min=13 * require a recent libsodium version or default to the one supplied * Increase the timeout for the resilience tests * Remove obsolete protocol-10 test code * Upgrade protocol for CAP-0034 / protocol issue 622 * Remove obsolete unsigned-nominated-value code * sets feeCharged with a min fee when checking validation * Transaction Queue sets feeCharged with a min fee when returning txINSUFFICIENT_FEE * force explicit use of potentially unsafe operations in uint128 * fix broken build * Add end-to-end test of checkpoint hash verification. * Document "verify-checkpoints" command and "--trusted-checkpoint-hashes" option to "catchup" * Support loading trusted checkpoint hashes from file in "catchup" command. * Add verify-checkpoints subcommand. * Add historywork/WriteVerifiedCheckpointHashesWork.cpp * Adjust hash-link propagation code in verifyHistoryOfSingleCheckpoint. * Rename getBufferedLedger to getFirstBufferedLedger, add getLastBufferedLedger. * Cancel timers in HerderImpl::shutdown to avoid spurious desyncs. * Add a new MODE_DOES_CATCHUP flag to Config (non-user-configurable) * git-ignore .idea (CLion) and .clangd * Set fail and/or badbit exception masks on fstreams. * Stop in-progress merges on shutdown * hide flaky resilience test * do not use temp variables in SQL queries * Command line: change up how force scp is used * change local history location in configs * add claimable balance to VS project files * add claimable balance to db-schema.md * update bucket test for ClaimableBalance * update BucketApplicator for ClaimableBalance * add tests for ClaimableBalance ops * fix invariants for ClaimableBalance * instantiate ClaimableBalance ops and pass index to create op * Create CreateClaimableBalanceOpFrame * Create ClaimClaimableBalanceOpFrame * fix invariant tests * add loadClaimableBalance to TransactionUtils * add XDR for claimable balance ops * Use claimable balance SQL in LedgerTxn * Implement SQL for ClaimableBalance * Update LedgerTxnTests for ClaimableBalance * Update LedgerHashUtils for ClaimableBalance * Update LedgerTestUtils for ClaimableBalance * Update LedgerEntryKey for ClaimableBalance * Refactor LedgerCmp and expand it for new LedgerEntryTypes * Add new LedgerEntry and LedgerKey types for ClaimableBalance * bump protocol version * Advance bucketlist level even for empty buckets * add SECTION so overlapping protocol versions don't conflict and cause Catch to skip test * increase catchup timeout in testcases for address sanitizer * Issue #2615: Add missing string to log message * log getpeername errors more explicitly * fix compile error - conform to C++ 14 * warning police * Remove version-checks for LedgerTxn after protocol 13 * Make SHA256 class stack-allocated, remove shared global gHasher. * Add "verify-hit benchmarking" test to CryptoTests.cpp * Use upper bound estimate for tx validity pre-consensus * Add option to use a close time offset for max time bound validity check * Add tests to ensure that the constant is used in both places and the meaning of the level is consistent across the codebase. * Introduce a constant for the maximum nesting level for quorums * run LedgerTxnTests with postgres * Test upgrades to database schema 13 * Live DB code for schema 13 with opaque extensions * DB schema upgrade to make extensions opaque * Maps between vectors and SQL records * Utilities for decoding base-64-encoded opaque XDR * Add a small test-code-injection interface * Fix ManageOfferOpFrameBase::insertLedgerKeysToPrefetch again * Collection of minor improvements * Add all data from StellarBeat to each node * Add all the node information available from survey results * Attach all edge information available * Start using PEP-0008 as our style guide ------------------------------------------------------------------- Wed Jul 01 08:07:51 UTC 2020 - marec@detebe.org - Update to version 13.2.0: * Add replacements urls * Clean up buckets in in-memory mode in ~BucketManagerImpl, not runCatchup. * take medida bump for missing include * Path payment test fixes * Use ms for consensus deltas logging * Bump tracy for RDTSC fix * Bump medida for #2518 (metrics overflow) and add test. * Add IP addresses and friendly names * Add bytes_transferred to each edge * Attach version information to each node * Fix the style using flake8 * bump min overlay version to 13 * Remove use of the term whitelist * Convert OverlaySurvey.py to Python 3 * bug 2573: support --metadata-output-stream command-line alternative to config. * Implement a spike feature for generateload * SCP: properly clear state during internal transition * bump tracy submodule to take warning fixes * bump medida submodule to take reporting fixes * Make print-xdr display account IDs properly * Cache the result of TxSetFrame::checkValid between calls. * Revert sqlite page size bump * Add tracy support to windows build * Broadly annotate: add ~600 tracy 'ZoneScoped's and various 'ZoneText's. * integrate lib/tracy/TracyClient.cpp into build * add tracy submodule ------------------------------------------------------------------- Fri May 29 17:24:12 UTC 2020 - marec@detebe.org - Update to version 13.1.0: * Fix formatting * Herder: change `ledgerClosed` to schedule nomination when we externalize * warning police * quiet down warnings from static analysis * use fmt submodule instead of vendored version * add libfmt submodule (6.2.1) * clarify build instructions for Ubuntu 16.04 * Centralize ledger validity calculation * Fix a typo in a command in testnet.md * Test out of order messages received by Herder * Allow Herder to process old externalize events * Fix max slots to remember math * recover from out of order externalized ledgers in CatchupManager * remove CloseLedgerIfResult * remove duplicate ledger header hash check * clang-format PendingEnvelopes, somehow it drifted * const-ify some Timer methods. * Update Visual Studio project * Rewrite VirtualClock::crank to use Scheduler, eliminate YieldTimer. * Add Scheduler class. * Switch VirtualClock::time_point to steady_clock, add system_time_point. * Add lib/util/finally.h * update MinGW instructions * Clear metrics when running simulation replay * herder: schedule `ledgerClosed` instead of calling "inline" * herder: adjust clamp down (was overly aggressive) * add an explicit error message when running into a crypto error * bump sqlite to 3.30.1 * update VS projects * warning police * Put transaction simulation files under txsimulation namespace * Tx simulation README * Command line: implement bucketlist and transaction simulation commands * Ledger replay: reduce allowed gap between replay and publishing * TxSimApplyTransactionsWork overhaul: extend functionality to support scaled ledger * SecretKey: implement operator< to allow using secret keys in std::set * Signature utils: allow signature verification with PublicKey * Implement simulation op frames for ManageSellOffer, ManageBuyOffer and CreatePassiveSellOffer * Implement simulated fee bump tx frame * Transaction bridge: add helper functions * Consistent naming: rename SimulationTxSetFrame * Consistent naming: rename SimulationTransactionFrame * Consistent naming: rename ApplyTransactionsWork * Consistent naming: rename SimulationMergeOpFrame * Introduce new works to simulate scaled bucketlist * Introduce a variety of tx simulation utility functions * Make toMuxedAccount utility function available outside of tests * Extract curr merge logic into a separate function * Allow sorting signers for AccountEntry * implement shutdown in ApplyBufferedLedgersWork * Command line publish: forget referenced buckets when finished ------------------------------------------------------------------- Thu Apr 30 10:02:16 UTC 2020 - marec@detebe.org - Update to version 13.0.0: * targeted scheduler fix: avoid throttling work when STANDALONE * Do not flood too many transactions at once * Do not flood invalid transactions after ledger close * remove redundant and extremely slow tests * bump catch version to 2.11.3 * overlay: retry a peer if it tells us that it has the information we need * Add comments. * Return nullptr from aborting WorkScheduler * Use a type that xdrpp-on-libc++ knows in Peer::pingIDfromTimePoint. * tests: explicitely set MANUAL_CLOSE * Make WorkScheduler::shutdown safe to call when already stopped * tests: don't roll virtual clock backwards * Reduce log frequency at INFO level of slow queries * workscheduler: throttle work getting executed * main queue: implement fail safe to avoid filling up memory when getting too much work * fix race in WriteSnapshotWork * overlay: delay call to shutdown * overlay: don't send empty peers list * overlay: bias towards low latency candidates when asking peers * add a timer to estimate latency between peers * fix tests that were making bad assumptions on queue * work scheduler uses the execution queue * separate io from actual work, remove concept of posting to next and current crank * adjust CAP0027 to use MuxedAccount for FeeBump * Fix another typo in comment * Fix typo in comment * Support CAP-0027 (first-class multiplexed accounts) from protocol 13 * clang-format police .x files * Revert "Fix ManageOfferOpFrameBase::insertLedgerKeysToPrefetch" * Fix bulk-load offers * Add bulk-load offer tests * Allow protocol upgrades in simulation * drop support for nodes that don't support surveying * bump overlay versions to min=11 cur=13 * warning police * ledger manager: use bucketized timer for "ledger.age.closed" * add new Buckets type to libmedida * bump libmedida * Fix posix parts broken by windows fixes. * Use random access handles when necessary on windows. * Fix signed/unsigned error in ranges. * Add CatchupRange.{cpp,h} to windows build * Tolerate EINTR better in IO operations, especially on XDR streams. * Override processSeqNum in SimulationTransactionFrame * wait on stellar-core to signal survey end * Fix issue with OverlayScript and remove unused json kvp * added new flag to maintain liabilities * change AllowTrustOp authorize flag to uint32 * only do auth checks from v9 or less * Remove support for TransactionMeta v1 * Update validation type * cap28 - always remove pre auth signers in tx apply * Replace existing removal of one-time signers with new method * Address review commits and rebase fallout. * tx queue: add instrumentation for time spent in queue * tx queue: add counter to track number of banned transactions * Add a TransactionQueue test for duplicate transaction that is not last * Fix TransactionQueue test that checks wrong semantics * Improve TransactionQueue removeAndReset and ban * Add test for boundary-landing replay, fix edge case. * Redo catchup ranges as half-open & separate bucket step. * Re-add support for special case of bucket-apply-only catchup N/0 * Remove incorrect special-case for CatchupRange starting on a checkpoint. * Add another helper to HistoryManager. * Remove now-dead {next,prev}CheckpointLedger * Use HistoryManager::ledgerToTriggerCatchup. * Add HistoryManager::ledgerToTriggerCatchup. * Rewrite CatchupManagerImpl::processLedger to avoid nextCheckpointLedger. * Remove more uses of prevCheckpointLedger. * Remove uses of {next,prev}CheckpointLedger, add comments. * Remove usage of nextCheckpointLedger and simplify surrounding code. * Add more helpers to HistoryManager.h * Remove use of prevCheckpointLedger and associated arithmetic chatter. * Add a couple comments about the existing methods we're retiring. * Remove uses of prevCheckpointLedger, getCheckpointFrequency. * Add more explicit ledger<->checkpoint calculation helpers to HistoryManager. * Replace a use of nextCheckpointLedger with publishCheckpointOnLedgerClose. * Replace a use of nextCheckpointLedger with checkpointContainingLedger. * Replace a use of nextCheckpointLedger with publishCheckpointOnLedgerClose. * Add more-obviously-named helper for deciding checkpoint-publish event. * Redefine checkpointContainingLedger to not use nextCheckpointLedger. * Add test for checkpointContainingLedger; remove for nextCheckpointLedger. * Delete comment mention of LedgerManager's currentLedger, focus on LCL. * Utilize catch warn NoTests to fail if a test does not match * Make test-partition files have trailing newline * make check fails if the number of tests is wrong * cap-0030 - remove NO_ISSUER results for all but changeTrustOp * stop prefetching issuer * Update VisualStudio project files * Make TxSetFrame SurgeCompare more like TransactionQueue canReplaceByFee * Fix warnings in TransactionQueue starting sequence boundary tests * Add overlay test to check that Floodgate::updateRecord prevents re-broadcast * Add TxSetFrame tests for fee-bump transactions * Add tests for upgrade to protocol 13 * Add TransactionQueue tests for fee-bump transactions and replace-by-fee * Add tests for FeeBumpTransactionFrame * Add CommandHandler tests for transaction envelope bridge and fee-bump submission * Update TxSetFrame for fee-bump transactions * Implement v0 to v1 bridge for CommandHandler tx endpoint * Improve comments and names in TransactionQueue * Update TransactionQueue on protocol 13 upgrade and avoid re-broadcasting transactions * Add updateRecord to Floodgate and updateFloodgateRecord to OverlayManager * Add TransactionQueue::maybeVersionUpgraded * Update TransactionQueue for fee-bump transactions * makeTransactionFromWire supports fee-bump transactions * Implement FeeBumpTransactionFrame * Do not overwrite meta in TransactionFrame::apply * Improve TransactionFrame::markResultFailed * Clean-up exception and txINTERNAL_ERROR handling in TransactionFrame * Add option to not charge fees in TransactionFrame * Remove transaction-related SQL from TransactionFrame and use free functions instead * Extract transaction-related SQL into free functions * Move makeTransactionFromWire to TransactionFrameBase * Use TransactionFrameBasePtr in herder tests and transaction queue tests * Use TransactionFrameBasePtr in closeLedgerOn * Use TransactionFrameBasePtr in LedgerManagerImpl and herder * Simplify prefetching in LedgerManagerImpl * Use TransactionFrameBase as the base-class for TransactionFrame * Add TransactionFrameBase * Do not use getResultPair * Fix transaction result tests for TransactionEnvelope v1 * Fix transaction envelope tests for TransactionEnvelope v1 * Fix transaction tests for TransactionEnvelope v1 * Fix herder tests for TransactionEnvelope v1 * Fix TxSetFrame for TransactionEnvelope v1 * Allow use of TransactionEnvelope v1 in transactionFromOperations * Allow use of TransactionEnvelope v1 in TransactionFrame * Add EnvelopeType validity checks to TransactionFrame * Use TransactionEnvelope v0 for fuzzer * Use TransactionEnvelope v0 for simulation * Use TransactionEnvelope v0 in HerderTests * Use TransactionEnvelope v0 in TxSetFrame * Use TransactionEnvelope v0 for transaction tests * Use TransactionEnvelope v0 in TransactionFrame and OperationFrame * Update signtxn for CAP-0015 * Add txbridge namespace containing functions for Transaction v0-v1 interop * Add XDR for CAP-0015 * Bump protocol version to 13 * fix catchup log message when catching up to current * Fix ManageOfferOpFrameBase::insertLedgerKeysToPrefetch ------------------------------------------------------------------- Tue Mar 24 19:35:20 UTC 2020 - marec@detebe.org - Update to version 12.5.0: * Collect prefetch hitrates per ledger in a histrogram * Track prefetch hitrate per ledger * fix DownloadBucketsWork progress status * Remove elapsed time from ConditionalWork * add more specific catchup logging * test infra: fix filtering of test cases * scp driver: add metrics to track externalize lag and delay * Add test for transaction queue starting sequence boundary * Improve TransactionQueue::toTxSet * scp driver: extract timing helper method * herder: centralize cleanup of slot information * Read merge counters after resolving futures * overlay: speed up and make resilience tests run during acceptance * fix crash in Simulation test (lifetime issue of LoopbackPeer) * always fully check SCP values when we have the corresponding local state * correctly reset catchup status * adjust test to ensure that slots stay fully validated on restart * add missing include (compile error) * warning police * README: Fix Stellar logo image * {Ballot,Nomination}Protocol::processEnvelope: Use early return to simplify flow * Do not fsync metadata output stream, pipes do not support. * remove delay in HistoryTestUtils on catchup failure * updates to CatchupManager logic * Fix (two different ways) unspecified tmpdir-teardown bug. * Interrupt quorum intersection checking on shutdown. * Make quorum intersection checking interruptable. * Start catchup from last buffered checkpoint on failure * stop buffering ledgers before a checkpoint * replace SyncingLedgerChain with deque * pick up latest libmedida (warning) * compile error (Windows) * Visual Studio project file fix * bump medida * Add tests for medida percentiles and sliding window reservoir sample * remove catchup state checks into LedgerManager * sync CatchupManager metrics * pull buffered ledgers from CatchupManager * remove catchup responsibilities from LedgerManager * add catchup responsibilities to CatchupManager * remove ProgressHandler mechanism from CatchupWork * remove unused LedgerManagerTests file ------------------------------------------------------------------- Wed Feb 26 18:24:54 UTC 2020 - marec@detebe.org - Update to version 12.4.0: * overlay: schedule the first write right away * overlay: change defaults for batch read operations * SCP: remove extra logging * set better defaults for sqlite databases * add missing doc on QUORUM_INTERSECTION_CHECKER config * overlay: add and expose batch write settings as config * overlay: expose batch read settings as config * herder: cleanup logging * overlay: cleanup logging * overlay: avoid degrading to async_read when we've done too much work already * fix typos * first commit - quick reference guide * overlay: fix typo that would cause the floodmap to be invalidated aggressively * bump current overlay version to 12 * dev container: added extra information on how to fix crlf interop * bump xdrpp submodule to absorb cereal fixes for writing json * Add --json flag to dump-xdr subcommand, implement via cereal * Move duplicate assetToString helper to util/types.h * removed empty file applicationtests.cpp * SCP: add missing reference to preparedPrime from envelope * SCP: log additional information when crashing * overlay: add entries in floodmap before processing SCP messages * overlay: add a method to remove entries from floodmap * overlay: add recvFloodedMsgID * perf cleanup * windows: use standard c++ recursive delete * Change straggler logic to use last-enqueued time, not last empty time. * Switch from buffered to vectored async_writes. * Use a larger read buffer and synchronously read as much as possible. * Add Async{Read,Write} metrics to OverlayMetrics * Add a YieldTimer helper class for starvation avoidance. * transaction queue: automatically ban transactions over quota * LedgerManager: centralize computation of ledger capacity * TransactionFrame: add a method to return the number of operations in a transaction * tx ban: allow to ban tx not in pool * allow 0 ops ledger upgrade * Add fetch duration metric per items * Make trace logs more detailed, less verbose * Remove forward-declaration of stellar::Config from SCP * Ignore any archive write commands in simulation mode * tests: make a single test folder per stellar-core instance * tests: make it explicit that we use the base instance for logging * tests: make the instance number explicitely part of the root folder * really remove pandoc dependency * Add instructions on how to work around shared workspace limitations * Use vscode account in dev container * Replace pandoc generated man-page with a static version * Add support for a VSCode Linux Container ------------------------------------------------------------------- Thu Jan 30 19:32:34 UTC 2020 - marec@detebe.org - Update to version 12.3.0: * PR changes * add overlay survey script * Document new overlay.delay.* metrics * Do not alter collation on the ledgerheaders.ledgerhash column, not worthwhile. * Add timers for queueing delays from send queue and asio queue. * absorb medida fix to exp-decay reservoir (thus: percentiles) in metrics * Cleanup db * properly apply buffered ledger when catching up * SCP: ensure that we really call nominatingValue with all new valid values * refresh pending envelopes metrics * fix shutdown * fix broken test: don't use wrapped value if we don't need to * make security release notes current as of 12.1.0 * herder: add tests for weak reference caches * herder: ensure that map learns new value introduced by combineCandidates * herder: add a map to track known txsets * herder: add a map to track known quorum sets * SCP: ballot protocol uses wrapped values * SCP: nomination uses wrapped values * Herder: attach quorum set and txsets to envelopes * Add SCPEnvelopeWrapper to SCP code * herder: only refresh metrics when converting to Json * PendingMessages: consolidate writes * Herder: cleanup PendingEnvelopes * properly return values referenced by BallotProtocol statements * avoid duplicate values returned for nomination statements * SCP: allow processCurrentState to always include self * SCP : replace low/high slot lookup by callback * SCP: replace getCurrentState by processCurrentState * move per slot cleanup to the same location * fix pending envelope tests: counter must be positive * tests: make rng seed non deterministic by default, fix missing command line support * update Catch to 2.11.1 * operations-per-ledger limit, instead of tx * Remove accountbalances index, no longer used outside full replay. * Add consensus simulation command line option * Make MAX_SLOTS_TO_REMEMBER configurable * Use pointer into unordered_map instead of iterator * ensure that operation level meta is cleared during an internal error * Implement MODE_ENABLES_BUCKETLIST * Remove AppMode, introduce switches in Config class instead * Bump sqlite page size to 64k. * fix Visual Studio project file (broken by cc6208b268d3102381c4e2502fcbb6e3afd4249a) * warning police * bump asio to 1.14.0 * Use simple "C" collation on varchar / text index columns. * Bump xdrpp to take fix for static destructor fiasco * bug 2318: print LCL on startup * Fix TxTest invariant - related to removing pre auth signers * herder: remove only remaining dynamic allocation in MinQuorumEnumerator. * util: add inline buffer to bitset. * [overlay] fix build on libstdc++ * add docs for topology survey * Add survey test cases * add survey endpoints * add medida counters and timers for survey messages * change PREFERRED_PEER_KEYS to a set of PublicKeys * rev overlay version for survey topology messages * add survey functionality to the overlay * Add ability to specify minimum overlay version on broadcast * return addRecord return value from recvFloodedMsg * add SurveyMessageLimiter * add PeerMetrics * Implemented encrypt/decrypt methods in Curve25519(formerly known as ECDH) * added xdr types for encrypted topology survey message * Fix LiabilitiesMatchOffers invariant * Fix bad file descriptor regex that only handles 1-digit fds, add logging. * Fix metadata live stream test to only stream from watcher-of-validators. * Missing piece for bug #2226, METADATA_OUTPUT_STREAM should deny NODE_IS_VALIDATOR * Have both docker containers in install instructions * Missing piece for bug #2226, need newdb in --replay-in-memory * Add basic round trip test of LedgerCloseMetaStream * Guard call to deleteObjectsModifiedOnOrAfter by modeHasDatabase() * Guard !modeHasDatabase startup against NODE_IS_VALIDATOR or any writable archives. * Add files to VS project files. * Add '--replay-in-memory' option to catchup subcommand. * Inhibit constructing database entirely when !modeHasDatabase. * Inhibit ExternalQueue and Maintenance when !modeHasDatabase * Store persistentState in memory when !modeHasDatabase. * Inhibit publishing when !modeHasDatabase. * Stream LedgerCloseMeta to XDR output stream. * Add XDROutputFileStream::{fdopen,flush}. * Virtualize AbstractTxSetFrame::toXDR, assert txs sorted. * Inhibit storing transactions and ledgers when !modeHasDatabase. * Delete buckets on shutdown when !modeHasDatabase * Use InMemoryLedgerTxnRoot + never-committed LedgerTxn for replay. * Add InMemoryLedgerTxnRoot.{cpp,h} * Allow use of different AbstractLedgerTxnParent types as LedgerTxnRoot. * Add Application::AppMode and use in Application implementations. * Add releaseAssertOr{Abort,Throw} release-assert helpers. * Make some static constants 'static const'. * warn if maintenance is set too low to keep up with network * fix typo in example config ------------------------------------------------------------------- Wed Nov 27 17:08:42 UTC 2019 - marec@detebe.org - Update to version 12.2.0: * bump overlay min version to 10 * doc: make README header engaging * warning police * update xdrpp to get rid of empty unions (which now cause clang warnings) * Windows build fix * Improve SQL for loadBestOffers to avoid quadratic complexity * Add offerid to bestofferindex * Prefetch when loading best offers * Implement load-by-doubling in LedgerTxnRoot::getBestOffer * Switch from list to deque in best offers cache and use std::upper_bound in findIncludedOffer * Update ApplyCheckpointWork and CatchupWork to wait for bucket merges asynchronously before applying ledgers * Create ApplyBufferedLedgersWork * Create ApplyLedgerWork * Remove applyBufferedLedgers from LedgerManagerImpl * expose buffered ledgers from LedgerManager * Improve getStatus message for ConditionalWork * Add methods to BucketList to allow us to check if buckets need to merge for a specific ledger * remove synchronous wait for merges * Add result verification flow to catchup * Update VS files * Tests for tx results verification * Add machinery to verify tx results * add test for bug 2314 * Rename NotWorseThan[] to LtEq[] to match <= operator, reformat comments. * Fix doubly-inverted relation definitions in LedgerTxnImpl.h comments. * Add comment and diagram about LedgerTxn::Impl::mWorstBestOfferMap. * Add comment about LedgerTxn::Impl::mMultiOrderBook. * Test that getBestOffer and loadBestOffer fail if there are active entries * Update LedgerTxn test specifier from ledgerstate to ledgertxn * Add in-memory order book tests * Use worst best offers for getBestOffer * Update interface for getBestOffer * Use in-memory order book for getBestOffer * Add in-memory order book to LedgerTxn::Impl * update github issue templates * add documentation for network upgrade expiration change * clean up floodgate map erase * Allow passing in archive name in catchup * Update example configuration file for SUPPORTED_META_VERSION * Update TransactionFrame to select between TransactionMetaV1 and TransactionMetaV2 * Add SUPPORTED_META_VERSION configuration flag * Update XDR for new transaction metaa ------------------------------------------------------------------- Fri Oct 25 15:45:44 UTC 2019 - marec@detebe.org - Update to version 12.1.0: * bug 2314: stop leaking buckets that are inputs to merges with empty outputs * Fix VS project * Visual C++ fix (filesystem is experimental in C++ 14) * Test catchup throttling * Throttle catchup when publish cannot keep up * Make helper functions in HistoryManager const * change test cases to intercept signal handler and exit on SIGINT * Stop referencing buckets during catchup when they are not needed * Abandon shared_future once next bucket is resolved * Update tests to reflect MergeKey changes * Remove maxProtocolVersion from MergeKey in merge re-attachment map * Rework finished merge re-attachment test to fail when merges are not attached correctly * Check apply-state validity only once per bucket-apply. * Use shared_ptr in BestOffersCache * Stop using std::string as the key for BestOffersCache * Remove per-key prefetch metrics * Correct spacing * Avoid clearing future when it is already clear * Make publish queue vs HAS more aggressive * Increase max depth to allow for Critical quality validators * Added Critical Quality level * Add non-serializing XDR hashers for SHA256 and ShortHash. * bug 2267: avoid focusing on largest SCC, pick any with a quorum. * herder: fix "SCP" DEBUG logging in tests * bitset: add explicit operator != * remove expired upgrades ------------------------------------------------------------------- Tue Oct 01 14:52:06 UTC 2019 - marec@detebe.org - Update to version 12.0.0: * disable inflation * Remove assert * Tests: ensure buckets are released from publish queue * docs: document the "critical" field in quorum and info endpoints. * Use all bucket hashes from HAS instead of relying on StateSnapshot for publish * fix windows project file * warning police * track peer total connection time * fix logging issues in overlay * bucket tests: avoid clever attempt at shutdown, use the normal Application dtor. * bucket tests: avoid blocking crank calls that race publish-process completion events. * bucket tests: avoid potential deadlocks reading merge counters with test or log locks held. * Added exchange tests for PATH_PAYMENT_STRICT_SEND * Update PathPaymentTests and add PathPaymentStrictSendTests * Clean-up extra includes in TxTests and PaymentOpFrame * Rename files PathPaymentOpFrame to PathPaymentStrictReceiveOpFrame * Rename PathPaymentOpFrame to PathPaymentStrictReceiveOpFrame * Add test machinery for PathPaymentStrictSend * Add PathPaymentStrictSendOpFrame * Refactor PathPaymentOpFrame to facilitate CAP-0024 * Add new rounding mode * bump overlay version * warning police * Fix rebuildLedgerFromBuckets command * Update BucketList tests to reflect shadow removal logic * Test publish and catchup after protocol upgrade * Test protocol 12 upgrade * Wait for merges to resolve after bucket application * Clear new-style future buckets for publish and validate buckets during catchup * Do not use shadows in new-style merges * Stop enforcing futures resolvedness in HAS and add utility functions * Bump protocol version * changed info endpoint to get quorum set of ledger last closed by the network if available * Windows: always setup the handle list when spawning processes * FileSystemException::getLastErrorString (windows) always include error code * export FileSystemException::getLastErrorString * Update name of LiabilitiesMatchOffers invariant * Correct asset url * work: add and tidy some comments. * transactions: add big comment about offers and the order book. ------------------------------------------------------------------- Thu Aug 29 17:30:36 UTC 2019 - marec@detebe.org - Update to version 11.4.0: * add forgotten virtual destructor to fuzzer base class * Windows: do not inherit all handles * Windows: improve filesystem error message * Windows: missing initialization * fix fuzzer integration with Windows * enable AFL_LLVM_MODE for afl-clang as part of configure.ac * update fuzzing.md with findings related to tx fuzzing work * move all fuzz code behind BUILD_TESTS in commandline * update makefile for fuzzer to minimize initial corpus and restrict memory limit * [fuzz general]: update makefile for new fuzzer cli args * [fuzz general]: simplify fuzz.cpp and utilize overlay and tx fuzzer impls * [fuzz general]: add createFuzzer method and add process_id & fuzzer mode args to cli * [fuzz tx]: autocheck to restrict public key range * [fuzz tx]: implement tx fuzzer * [fuzz tx]: create FuzzTransactionFrame and define a resetTxInternalState method for fuzzer * [fuzz overlay]: implement overlay fuzzer * [fuzz general]: create base fuzzer class * [fuzz general]: add resetForFuzzer method to LedgerTxnRoot * [fuzz general]: allow fuzzer to ignore signature checks * reduce the limit of open descriptors in travis * fix: use MAX_PENDING_CONNECTIONS in config as upper bound * Remove stream cleanup from ApplyCheckpointWork, let parent work (BatchWork) handle cleanup * Set max number of file descriptors in Travis build * Added ConditionalWork test that verified child works are not kept alive * remove obsolete file from VS studio project * warning police * Explicitly close files in ApplyCheckpointWork * Reset condition once satisfied, test pass * improve parallel build instructions * fix issue where the final sed command can fail and exit with error code 2 * Sort metrics by name in metrics.md * reappend overriden -std=c++14 for enable-afl in configure.ac * Add a bit more explanation on the last test. * bug 2234: scrub passwords from connection strings better. * increase maximim nesting level to allow for HIGH -> LOW validators * move quorum sanity check to config validation * write testnet config using HOME_DOMAINS (recommended way) * bug 2123: do not try to (confusingly) "repair" missing buckets, just fail. * Cantralize status logging in CatchupWork * Update VS files * Avoid logging catchup progress in BatchWork as BatchWork is a general-purpose work * Switch history replay mechanism to batching of download-apply works * Update ApplyCheckpointWork to apply a single checkpoint * fix max size of base64 encoded columns * Rename ApplyLedgerChainWork to ApplyCheckpointWork * Introduce ConditionalWork * make it easier to manage min/max schema versions * squash database schema version 9 * squash database schema version 8 * squash database version 7 * ProcessManager: redirect process output to a temp file, rename at the end * Test publishing to multiple archives * Refactor publish-related works to avoid race-conditions * Avoid populating has by reference in GetHistoryArchiveStateWork, instead query explicitly when needed * Make work utility functions more general-purpose * add missing <algorithm> include * Add reference to HISTORY attribute when configuring archives * Clarify that NODE_HOME_DOMAIN needs to be set * bug 2202: fsync XDR on close (also bucket dir), with option to disable. * move some simulation tests to acceptance category to not be run on travis * move some overlay tests to acceptance category to not be run on travis * move some herder tests to acceptance category to not be run on travis * move some invariant tests to acceptance category to not be run on travis * add information about running acceptance tests * redo history test categories * move some tests to acceptance category to not be run on travis * Fail loadgen when load is too high, remove Simulation dependency * Improve naming consistency of federated voting functions * Update CONTRIBUTING. * docs: add note to admin guide about snapshotting databases post meta-data generation * Update docs/db-schema.md * fix banning nodes ------------------------------------------------------------------- Thu Jul 25 15:32:19 UTC 2019 - marec@detebe.org - Update to version 11.3.0: * use less resources when running tests * keep transactions in transaction queue when removing txs that were processed in a ledger * warning police * add missing files to visual studio project * bucket: fix comment typo. * bucket: add tests for reattaching to merge futures. * history: add some testing interfaces. * bucket: add machinery to reattach to / resynthesize merge futures. * bucket: add test for BucketMergeMap type. * bucket: add BucketMergeMap type to record weak input/output merge mappings. * bucket: add MergeKey type for recording and fetching merges. * bucket: replace some insert() calls with emplace() calls. * bucket: add MergeCounters::operator== * add "fullkeys" support to "peers" endpoint * make Config::toStrKey switch between full/short keys * rename Config::toStrKey to Config::toStrKeyAuto to remove ambiguity * Simplify SimulationTxSetFrame::getBaseFee * ApplyTransactionsWork controls max transaction set size * Run simulation from the command line * Abstract TxSetFrame into AbstractTxSetFrameForApply and implement SimulationTxSetFrame * fix windows compile: medida::TimerContext is not copyable * Audit Work retries * BasicWork: reduce delay cap and remove unsused RETRY_FOREVER * Make TransactionFrame polymorphic for simulation and implement SimulationTransactionFrame * Implement HistoryArchiveStream * Add new metrics to metrics.md * Fix joined row in metrics.md * Add primary performance metrics * Add new Perf logging namespace * redo pending transactions * Make travis happy: clang format, afl llvm mode behind an ifdef, clearing queue fix * history: add test for FutureBucket eager resolution in BucketList. * update all docs and files with new command line syntax * history: add a couple comments about resolved-ness at serialization time. * bucket: sink resolveAnyReadyFutures from HAS at ledger close to addBatch. * buckets: add 'BucketList::resolveAnyReadyFutures'. * history: remove dead 'HistoryArchiveState::futuresAllReady'. * herder: add test of intersection-criticality checker. * herder: wire up intersection-criticality checker to info & quorum endpoint. * herder: add getIntersectionCriticalGroups intersection-criticality checker. * herder: Add "quiet" mode to QuorumIntersectionChecker for subroutine use. * filter out invalid messages from overlay fuzzer * seed randomness in overlay fuzz * use simulation to coordinate two overlay fuzz apps & remove cfg dir * add getLoopbackPeerConnection method to Simulation * add clearInAndOutQueues method to LoopbackPeer * remove REQUIRE macro and instead throw so shutdownWorkScheduler can be used by fuzzer * allow for deterministic randombytes_buf generation * Update Makefile to work with 'new' commandline * fix coverity scan warnings * Fix drop buckets after start new ledger, introduced by d05a42d3 * Fix make check * stellar-core has result 1 if it displays usage information when not explicitly requested * Update admin.md * Update admin.md * herder: order qmap before hashing, reduce spurious recalculation. * improve error messages during catchup * buckets: add bucket-hash faking, for varying bucket logic in test replay. * catchup: tighten maintenance defaults to avoid huge db during catchup. * buckets: add rebuild-ledger-from-buckets, move load-xdr to BUILD_TESTS. * warning police * improve error message when locking fails * fix catchup current/0 error on very up-to-date databases ------------------------------------------------------------------- Thu Jun 27 17:43:33 UTC 2019 - marec@detebe.org - Update to version 11.2.0: * add 'parallel' to packages, re #2025 * Remove redundant -- one might say "straggling" -- copy of straggler metric. * Update VS project files. * bucket merge: add context to exception thrown when merging buckets * main: properly log exception message when crashing * Travis: provide some values to ccache * overlay: cache metrics so we don't keep looking them up. * travis: remove bogus ".libs" in ccache configuration ; add missing llvm-5 package add logging in compile script * VS project: move github related files to its own filter * Move quorum intersection checker from scp to herder. * Add missing dependency for running tests on mac * warning police * fix compile error (VC++) * fix VS 2019 project files * some spelling nits -- thanks to 'Code Spell Checker' * Remove / from http-command examples in docs * Don't look up http command handler twice * remove deprecated command line handler * update preconditions for CatchupRange constructor * config: add QUORUM_INTERSECTION_CHECKER to control check. * quorum checking: support history and/or present checking via command-line args. * config: bump WORKER_THREADS by 1 * herder: report /info quorum status differently, include intersection. * herder: continuously monitor quorum intersection. * history: use new quorum intersection and herder qtracker for check-quorum. * tests: adapt and expand quorum intersection tests for new checker. * SCP: add new quorum intersection checker, with long comment. * herder: expose getCurrentlyTrackedQuorum to clients. * history: add InferredQuorum::getQuorumMap. * herder: rebuild quorum tracker state immediately on restore. * herder: factor out rebuildQuorumTrackerState. * util: make rand_flip() much cheaper. * util: add BitSet utility type. * do not allow gaps in history tables * Clean up warnings in tests * Warning police * Config: compute a better value for FAILURE_SAFETY (automatic quorum set) * Config: add automated quorum generation test * Config: migrate testnet config to automated quorum set configuration * Config: add documentation for auto quorum set * Config: add NODE_HOME_DOMAIN, automatically add node to quorum set * Config: check that auto quorum sets have enough archives * Config: add warnings/status messages when mixing auto quorum set with direct mode * Config: add automatic quorum set generator code * Config: add parsing code for HOME_DOMAINS * Config: add parsing code for VALIDATORS * Config: extract addValidatorName * Config: extract addHistoryArchive * Config: extract computeDefaultThreshold to compute thresholds * Config: refactor processConfig * expose LocalNode::toJson using lambdas * Improve quorum set normalization * clarify standalone configuration comments * Upgrade medida to expose count metric in histogram and timer * Persist SCP state for previous slots * Added v11.0.0 changes to release notes * Add new tests for invalid upgrades * Check validity of upgrades before applying them during closeLedger * Remove default issue template. * Update issue templates * warning police * warnings (windows) * update Windows readme to VS2019 * update projects to vs2019 * snap to latest xdrpp * snap to libsodium v1.0.18 * admin.md spelling nit * Add SCP timeout metrics * Fix formatting of issue template. * Clean up issue template. * Improve tests and offline catchup * Include algorithm in NominationProtocol.cpp :( * fix logging issue:could not set global log level via http * Added SECURITY.md * Do not retry repair missing buckets work * Fix url ------------------------------------------------------------------- Fri May 24 07:57:26 UTC 2019 - marec@detebe.org - Update to version 11.1.0: * bump min overlay to 8 * implement upgrade-db command * display known keys when displaying short keys * Add transitive quorum information as an option to the quorum endpoint * Simplify catchup manual test * various cleanup * changes on /quorum?fullkeys=true * herder: make non validating nodes behave more like regular nodes * Implicitly inline InvariantDoesNotHold * Do not crash Work when dealing with bad files * add a build stage to travis runs before running tests * make it possible to invoke travis build script to not run tests * build script: parametrize the build script better * Add diagram comment for BasicWork state machine. * test: break down saturated node test into deterministic steps * simulation: make startAllNodes idempotent * warning police: avoid passing large objects by value * warning police: uninitialized variables * Make "ll" endpoint properly deal with mismatch in partition case * logging: expose logging partitions * Herder: don't start "in sync" if we're on genesis ledger * Herder: check close time as soon as we receive messages * Herder: make checkCloseTime helper function public * Herder: discard SCP messages that are unlikely to be from the latest ledger (when joining the network) * HerderSCPDriver: add method to convert a Value into a StellarValue * add MAXIMUM_LEDGER_CLOSETIME_DRIFT configuration * Fix Windows process related bug * Update VS files * fix compile issue (vs2019) * update libmedida to use a version that compiles with vs2019 * update asio to 1.13.0 * update cpptoml to 1.1 * Disallow parent work to succeed if a child failed * bugfix: use children status directly when publishing snapshot * added missing docs files in VS project * Update WorkScheduler comment and return log * Remove obsolete include * Update remaining tests to utilize WorkScheduler * Refactor work test suite to consider possible failure modes * Update Application to employ WorkScheduler, and wait for its proper shutdown * Refactor FetchRecentQset work to comply with the new interface * Tweak ProcessManager to allow Work to shutdown processes during abort * Refactor publish-related works and update orchestration in HistoryArchiveManager to comply with the new interface * Implement RepairMissingBuckets work * Remove BucketDownloadWork as catchup does not use it anymore * Refactor catchup-related works to comply with the new interface * Refactor get history archive state work to comply with the new interface * Refactor RunCommandWork and its implementers to comply with the new interface * Introduce work building blocks * Move BatchWork to work subfolder ------------------------------------------------------------------- Thu Apr 25 19:37:27 UTC 2019 - marec@detebe.org - Update to version 11.0.0: * fix AssetCode12 to actually be up to 12 chars * Typo and small edits to Bucket README. * fix broken build - clang format was not run before merging * add missing schema documentation for new table `quoruminfo` * tidy up XDR types somewhat: - Use enum type for union discriminant that needs it - Typedef asset codes because they have additional constraints * better log msgs * warning police * add missing files to Visual Studio project * clearer command line deprecation ------------------------------------------------------------------- Tue Mar 19 19:50:11 UTC 2019 - marec <marec@detebe.org> - initial package build
Locations
Projects
Search
Status Monitor
Help
OpenBuildService.org
Documentation
API Documentation
Code of Conduct
Contact
Support
@OBShq
Terms
openSUSE Build Service is sponsored by
The Open Build Service is an
openSUSE project
.
Sign Up
Log In
Places
Places
All Projects
Status Monitor
