What's new in Kopia 0.16.1
Mar 26, 2024
- Release Highlights:
- New in v0.16.1: update UI to the latest version - #3748
- Azure PIT support (#3407) by Mike McKay-Dirden
- [EXPERIMENTAL] Implement volume shadow copy support on Windows (#3543) by Maxim Khitrov
- UI: Add the ability to change font-sizes (#3515) by Christoph Anderson
- [BREAKING CHANGE] Disable legacy API by default (#3730) by Nick - See #3716 - this is in preparation to completely remove old non-GRPC repository API in kopia v0.17.0.
- allow disabling writes on index index loads (#3645) by Julio López
- Change Log:
- Command-Line Interface:
- New Feature 1B: PPROF extensions: load profile config (#3668) by Aaron Alpar
- New Feature 1a: PPROF extensions: PEM dump (#3665) by Aaron Alpar
- New Feature handle SIGTERM (#3562) by Julio Lopez
- New Feature A: Add ability to dump pprof data to logs ... (#3454) by Aaron Alpar
- Output error msg returned by API server payload when possible (#3724) by Nick
- minor cleanups in server start (#3637) by Julio López
- server shutdown (#3608) by Julio López
- miscellaneous code simplifications (#3607) by Julio López
- Ensure scrubber checks all fields recursively for sensitive tags (#3589) by Nick
- print errors during processing in red (#3514) by André Schröder
- Don't return error when parameters unchanged (#3411) by Mike McKay-Dirden
- Graphical User Interface:
- New Feature Add the ability to change font-sizes (#3515) by Christoph Anderson
- KopiaUI App:
- Add timeout in pollOnce https.request equal to poll interval (#3055) by Till Riedel
- General Improvements:
- rename var to hasEpochManager for clarity (#3744) by Julio López
- add epoch.Manager.MaybeGenerateRangeCheckpoint (#3727) by Julio López
- add epoch.Manager.CleanupMarkers (#3726) by Julio López
- add epoch.Manager.MaybeCompactSingleEpoch (#3728) by Julio López
- closedIntRange (#3735) by Julio López
- add clarification comment (#3723) by Julio López
- add epoch.getKeyRange helper (#3721) by Julio López
- add epoch.Manager.MaybeAdvanceEpoch helper (#3709) by Julio López
- Increase timeout to check for generated TLS cert (#3696) by chaitalisg
- fix TestIndexEpochManager_CompactionAlwaysFails (#3677) by Julio López
- remove unused context.Context field (#3664) by Julio López
- allow disabling writes on index index loads (#3645) by Julio López
- move index cleanup out to refreshAttempt (#3603) by Julio López
- add TestSortAndDedupeTimesOfDay (#3535) by Julio Lopez
- Delete duplicates in time of day array within policies (#3484) by Christoph Anderson
- use require.Eventually to ensure test stops (#3530) by Julio Lopez
- fix race in TestIndexEpochManager_NoCompactionInReadOnly (#3517) by Julio Lopez
- minor cleanups related to index managers (#3518) by Julio Lopez
- minor cleanups (#3495) by Julio Lopez
- cleanup nits (#3481) by Julio Lopez
- correct spelling by Aaron Alpar
- Repository:
- New Feature Add Azure PIT support (#3407) by Mike McKay-Dirden
- New Feature Add support for Azure DeleteBlob operations where immutability protection is on (#3394) by Mike McKay-Dirden
- added metrics timeseries computation (#3678) by Jarek Kowalski
- added context to potentially blocking repository methods (#3654) by Jarek Kowalski
- refactor internal/repolog package to support writing metrics in the future (#3610) by Jarek Kowalski
- Server:
- Return internal server errors for Content Get errors (#3713) by Nick
- do not make blocking calls inside server status API (#3666) by Jarek Kowalski
- Snapshots:
- Experimental New Feature Implement volume shadow copy support on Windows (#3543) by Maxim Khitrov
- Fixup for #3624. Shadow copy seems to extend time for snapshots. (#3625) by Aaron Alpar
- Storage Providers:
- change versioning check for Azure storage (#3520) by Mike McKay-Dirden
- New Feature support for Azure immutability protection (#3412) by Mike McKay-Dirden
- New Feature Add ExtendBlobRetention support for Azure (#3405) by Mike McKay-Dirden
- add variables for Azure immutability testing (#3471) by Julio Lopez
- fix documentation for S3 permissions (#3451) by Julio Lopez
- CI/CD:
- upgrade linter to 1.56.2 (#3714) by Jarek Kowalski
- install apple notarization key before build (#3715) by Jarek Kowalski
- fix shadow-copy test (#3629) by Aaron Alpar
- fixed HTMLUI E2E test by switching to macOS runner (#3613) by Jarek Kowalski
- upgrade gotestsum to 1.11.0 (#3534) by Julio Lopez
- upgrade workflows to work with actions/{upload,download}-artifact@v4 (#3532) by Julio Lopez
New in Kopia 0.16.0 (Mar 24, 2024)
- Release Highlights:
- Azure PIT support (#3407) by Mike McKay-Dirden
- [EXPERIMENTAL] Implement volume shadow copy support on Windows (#3543) by Maxim Khitrov
- UI: Add the ability to change font-sizes (#3515) by Christoph Anderson
- [BREAKING CHANGE] Disable legacy API by default (#3730) by Nick - See #3716 - this is in preparation to completely remove old non-GRPC repository API in kopia v0.17.0.
- allow disabling writes on index index loads (#3645) by Julio López
- Change Log:
- Command-Line Interface:
- New Feature 1B: PPROF extensions: load profile config (#3668) by Aaron Alpar
- New Feature 1a: PPROF extensions: PEM dump (#3665) by Aaron Alpar
- New Feature handle SIGTERM (#3562) by Julio Lopez
- New Feature A: Add ability to dump pprof data to logs ... (#3454) by Aaron Alpar
- Output error msg returned by API server payload when possible (#3724) by Nick
- minor cleanups in server start (#3637) by Julio López
- server shutdown (#3608) by Julio López
- miscellaneous code simplifications (#3607) by Julio López
- Ensure scrubber checks all fields recursively for sensitive tags (#3589) by Nick
- print errors during processing in red (#3514) by André Schröder
- Don't return error when parameters unchanged (#3411) by Mike McKay-Dirden
- Graphical User Interface:
- New Feature Add the ability to change font-sizes (#3515) by Christoph Anderson
- KopiaUI App:
- Add timeout in pollOnce https.request equal to poll interval (#3055) by Till Riedel
- General Improvements:
- rename var to hasEpochManager for clarity (#3744) by Julio López
- add epoch.Manager.MaybeGenerateRangeCheckpoint (#3727) by Julio López
- add epoch.Manager.CleanupMarkers (#3726) by Julio López
- add epoch.Manager.MaybeCompactSingleEpoch (#3728) by Julio López
- closedIntRange (#3735) by Julio López
- add clarification comment (#3723) by Julio López
- add epoch.getKeyRange helper (#3721) by Julio López
- add epoch.Manager.MaybeAdvanceEpoch helper (#3709) by Julio López
- Increase timeout to check for generated TLS cert (#3696) by chaitalisg
- fix TestIndexEpochManager_CompactionAlwaysFails (#3677) by Julio López
- remove unused context.Context field (#3664) by Julio López
- allow disabling writes on index index loads (#3645) by Julio López
- move index cleanup out to refreshAttempt (#3603) by Julio López
- add TestSortAndDedupeTimesOfDay (#3535) by Julio Lopez
- Delete duplicates in time of day array within policies (#3484) by Christoph Anderson
- use require.Eventually to ensure test stops (#3530) by Julio Lopez
- fix race in TestIndexEpochManager_NoCompactionInReadOnly (#3517) by Julio Lopez
- minor cleanups related to index managers (#3518) by Julio Lopez
- minor cleanups (#3495) by Julio Lopez
- cleanup nits (#3481) by Julio Lopez
- correct spelling by Aaron Alpar
- Repository:
- New Feature Add Azure PIT support (#3407) by Mike McKay-Dirden
- New Feature Add support for Azure DeleteBlob operations where immutability protection is on (#3394) by Mike McKay-Dirden
- added metrics timeseries computation (#3678) by Jarek Kowalski
- added context to potentially blocking repository methods (#3654) by Jarek Kowalski
- refactor internal/repolog package to support writing metrics in the future (#3610) by Jarek Kowalski
- Server:
- Return internal server errors for Content Get errors (#3713) by Nick
- do not make blocking calls inside server status API (#3666) by Jarek Kowalski
- Snapshots:
- Experimental New Feature Implement volume shadow copy support on Windows (#3543) by Maxim Khitrov
- Fixup for #3624. Shadow copy seems to extend time for snapshots. (#3625) by Aaron Alpar
- Storage Providers:
- change versioning check for Azure storage (#3520) by Mike McKay-Dirden
- New Feature support for Azure immutability protection (#3412) by Mike McKay-Dirden
- New Feature Add ExtendBlobRetention support for Azure (#3405) by Mike McKay-Dirden
- add variables for Azure immutability testing (#3471) by Julio Lopez
- fix documentation for S3 permissions (#3451) by Julio Lopez
- CI/CD:
- upgrade linter to 1.56.2 (#3714) by Jarek Kowalski
- install apple notarization key before build (#3715) by Jarek Kowalski
- fix shadow-copy test (#3629) by Aaron Alpar
- fixed HTMLUI E2E test by switching to macOS runner (#3613) by Jarek Kowalski
- upgrade gotestsum to 1.11.0 (#3534) by Julio Lopez
- upgrade workflows to work with actions/{upload,download}-artifact@v4 (#3532) by Julio Lopez
New in Kopia 0.15.0 (Oct 22, 2023)
- Graphical User Interface:
- fixed hint text visibility in dark mode (#3393) by Jarek Kowalski
- Fixing colors in dark theme (#3392) by Jarek Kowalski
- Command-Line Interface:
- Fixing help text when editing policies externally (#3357) by Christoph Anderson
- Convert --run-missed from bool to Enum (#3337) by PhracturedBlue
- Attempt to fix socket-activated tests (#3318) by PhracturedBlue
- KopiaUI App:
- Fix missing window when external displays are detached (#3336) by Christoph Anderson
- Repository:
- fixed handling of content.Info (#3356) by Jarek Kowalski
- added fs.DirectoryIterator (#3365) by Jarek Kowalski
- Move encryption into a common package in preparation to handle secrets (#3351) by PhracturedBlue
- Server:
- New Feature improve scheduler algorithm to run missed snapshots (#3323) by PhracturedBlue
- Snapshots:
- Minor reorder of defer funcs for streaming files (#3364) by ashmrtn
- Name policy tests to make debugging easier (#3332) by PhracturedBlue
- Dependencies:
- bump github.com/chromedp/chromedp from 0.9.2 to 0.9.3 (#3391) by dependabot[bot]
- bump golang.org/x/net from 0.16.0 to 0.17.0 (#3382) by dependabot[bot]
- bump ossf/scorecard-action from 2.2.0 to 2.3.0 (#3378) by dependabot[bot]
- bump github/codeql-action from 2.21.9 to 2.22.1 (#3377) by dependabot[bot]
- bump the common-golang-dependencies group with 8 updates (#3376) by dependabot[bot]
- bump the kopia-ui-npm-dependencies group (#3366) by dependabot[bot]
- bump the common-golang-dependencies group with 3 updates (#3370) by dependabot[bot]
- bump github/codeql-action from 2.21.8 to 2.21.9 (#3369) by dependabot[bot]
- bump github/codeql-action from 2.21.7 to 2.21.8 (#3355) by dependabot[bot]
- bump the common-golang-dependencies group with 2 updates (#3353) by dependabot[bot]
- bump actions/checkout from 4.0.0 to 4.1.0 (#3354) by dependabot[bot]
- bump docker/setup-qemu-action from 2 to 3 (#3324) by dependabot[bot]
- bump go.uber.org/zap from 1.25.0 to 1.26.0 (#3328) by dependabot[bot]
- bump github.com/klauspost/compress from 1.16.7 to 1.17.0 (#3329) by dependabot[bot]
- bump the common-golang-dependencies group with 9 updates (#3327) by dependabot[bot]
- bump github/codeql-action from 2.21.5 to 2.21.7 (#3325) by dependabot[bot]
- Development Dependencies:
- bump electron from 26.1.0 to 26.2.1 in /app (#3320) by dependabot[bot]
New in Kopia 0.14.1 (Sep 18, 2023)
- Command-Line Interface:
- New Feature Added option to allow running missed snapshots on Kopia start (#3311) by PhracturedBlue
- Graphical User Interface:
- fixed dark mode via kopia/htmlui#191 (#3314) by Jarek Kowalski
- KopiaUI App:
- Workaround for wrong window bounds in multi display environments (#3315) by Christoph Anderson
- CI/CD:
- auto-delete old RPM and DEB packages (#3310) by Jarek Kowalski
New in Kopia 0.14.0 (Sep 16, 2023)
- Graphical User Interface:
- Changed default tab to Snapshots (#3308) by Jarek Kowalski
- New Feature Allow specifying whether local or all snapshots are shown by default in the UI (#3289) by PhracturedBlue
- New Feature imported latest UI changes (#3272) by Jarek Kowalski
- Fixed error counter reported on task level (#3190) by Christoph Anderson
- New Feature UI updates (Theming support, action fix) (#3145) by Jarek Kowalski
- Command-Line Interface:
- Added output to indicate completion of snapshot verification (#3275) by Jarek Kowalski
- New Feature added '--otlp-trace' flag that replaces Jaeger tracing with OTLP over gRPC (#3264) by Jarek Kowalski
- Minor nits for observability flags (#3209) by Julio Lopez
- New Feature save observability metric before exiting the process (#3201) by Julio Lopez
- Break down observability.startMetrics() (#3196) by Julio Lopez
- Correct typo in blob stats command (#3194) by Julio Lopez
- Prevent duplicate snapshots when --all and a source path are given (#3067) by NickIAm
- Log cache full when on debug level (#3085) by Aurélien Rouëné
- New Feature new storage config input from file when creating a repo (#2756) by Shikhar Mall
- Preserve error exit code when '--json' output is specified (#3163) by Julio Lopez
- New Feature add --override-source parameter to allow overriding source when snapshotting (#3041) by Maxim Schuwalow
- KopiaUI App:
- Fixed opening file browser for mounted directories (#3307) by Jarek Kowalski
- New Feature Retain window size and location after close (#3302) by Christoph Anderson
- General Improvements:
- Fixed TestOnNthCompletion test flake, which was 1-5/10000 runs (#3263) by Jarek Kowalski
- Remove unnecessary //nolint directives (#3256) by Julio Lopez
- Add checklocks annotation (#3247) by Julio Lopez
- Rename cache variable in test (#3211) by Julio Lopez
- Change cache full logging frequency & message (#3193) by Julio Lopez
- Add crash consistency test to validate repository consistency when kopia stops abruptly (#3142) by Minyi Zou
- Cleaner error checking in retention tests (#3164) by ashmrtn
- Minor followups to #3041 (#3141) by Julio Lopez
- Cleanup "stale" error naming (#3129) by Julio Lopez
- Output content rewritten bytes to INFO (#3061) by NickIAm
- Handle no space left errors in robustness tests (#3043) by chaitalisg
- Repository:
- Removed useless log (#3299) by Jarek Kowalski
- New Feature apply retention policies server-side (#3249) by Jarek Kowalski
- New Feature added hard size limit to the on-disk cache (#3238) by Jarek Kowalski
- New Feature allow configuring auto compaction count (#3240) by Vaibhav Kamra
- Disable manifest manager compaction when in read-only mode (#3226) by ashmrtn
- Skip epoch compaction and advancement on read-only storage (#3225) by ashmrtn
- More precise check for retention in tests (#3220) by ashmrtn
- Storage config blob caching (#3192) by ashmrtn
- Fix s3 list bucket permission error (#3072) by lyndon
- New Feature Implement retention time extension on S3 buckets using Object Locks (#2179) by PhracturedBlue
- Server:
- New Feature reduce server refreshes of the repository (#3265) by Jarek Kowalski
- New Feature introduced generic scheduler (#3293) by Jarek Kowalski
- New Feature Add socket-activation support (#3283) by PhracturedBlue
- New Feature fixed server logging file & console (#3262) by Jarek Kowalski
- New Feature add server support for unix domain sockets (#3216) by PhracturedBlue
- Snapshots:
- New Feature added ability to use cron expressions to schedule snapshots (#3149) by Jarek Kowalski
- Storage Providers:
- Fixed list/get caching with rclone providers (#3284) by Jarek Kowalski
- New Feature allow other GCS OAuth2.0 credential types (#3064) by Le Tran
- Create default provider that has common functions (#3241) by ashmrtn
- New Feature Create IsReadOnly API for blob storage (#3230) by ashmrtn
- New Feature improve performance of SFTP uploads (#3235) by Jarek Kowalski
- New Feature support for auth with Azure AD (#3131) by Wenkai Yin(尹文开)
- Use target ref in repo checkout (#3159) by Julio Lopez
- Split providers test workflow (#3158) by Julio Lopez
- Use minio SDK credentials package in S3 STS test (#3148) by Julio Lopez
- Dependencies:
- Use commit id for GH action versions (#3298) by Julio Lopez
- bump ossf/scorecard-action from 2.1.2 to 2.2.0 (#3291) by dependabot[bot]
- bump github/codeql-action from 2.2.4 to 2.21.5 (#3292) by dependabot[bot]
- bump actions/checkout from 3.1.0 to 4.0.0 (#3290) by dependabot[bot]
- bump actions/checkout from 3 to 4 (#3271) by dependabot[bot]
- bump github.com/google/fswalker from 0.3.0 to 0.3.2 (#3268) by dependabot[bot]
- bump the common-golang-dependencies group with 4 updates (#3273) by dependabot[bot]
- bump github.com/hanwen/go-fuse/v2 from 2.3.0 to 2.4.0 (#3270) by dependabot[bot]
- bump the common-golang-dependencies group with 4 updates (#3242) by dependabot[bot]
- bump github.com/google/uuid from 1.3.0 to 1.3.1 (#3243) by dependabot[bot]
- bump github.com/pkg/sftp from 1.13.5 to 1.13.6 (#3222) by dependabot[bot]
- bump the common-golang-dependencies group with 2 updates (#3221) by dependabot[bot]
- bump the common-golang-dependencies group with 4 updates (#3198) by dependabot[bot]
- bump go.uber.org/zap from 1.24.0 to 1.25.0 (#3199) by dependabot[bot]
- bump github.com/chromedp/chromedp from 0.9.1 to 0.9.2 (#3200) by dependabot[bot]
- bump the kopia-ui-npm-dependencies group (#3188) by dependabot[bot]
- bump the common-golang-dependencies group with 2 updates (#3186) by dependabot[bot]
- bump the common-golang-dependencies group with 1 update (#3173) by dependabot[bot]
- have dependabot ignore kopia/htmluibuild updates (#3166) by Julio Lopez
- bump the common-golang-dependencies group with 1 update (#3160) by dependabot[bot]
- add Azure to common-golang dependabot group (#3155) by Julio Lopez
- bump the kopia-ui-npm-dependencies group (#3156) by dependabot[bot]
- bump github.com/Azure/azure-sdk-for-go/sdk/azcore (#3152) by dependabot[bot]
- bump github.com/Azure/azure-sdk-for-go/sdk/storage/azblob (#3153) by dependabot[bot]
- bump the common-golang-dependencies group with 2 updates (#3151) by de...
New in Kopia 0.13.0 (May 2, 2023)
- General Improvements:
- added tests for parallelwork (#2942) by Denis Voytyuk
- added unit tests for diff (#2931) by Denis Voytyuk
- added tests for iocopy (#2932) by Denis Voytyuk
- added tests for tlsutil (#2933) by Denis Voytyuk
- fixed remaining checklocks violations (#2939) by Jarek Kowalski
- added unit tests for freepool (#2930) by Denis Voytyuk
- Repository:
- New Feature Reduce memory usage when parsing manifests (#2956) by ashmrtn
- Dependencies:
- bump github.com/aws/aws-sdk-go from 1.44.252 to 1.44.253 (#2981) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.251 to 1.44.252 (#2979) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.250 to 1.44.251 (#2977) by dependabot[bot]
- bump google.golang.org/api from 0.119.0 to 0.120.0 (#2975) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.249 to 1.44.250 (#2972) by dependabot[bot]
- bump github.com/hanwen/go-fuse/v2 from 2.2.0 to 2.3.0 (#2966) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.247 to 1.44.249 (#2965) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.246 to 1.44.247 (#2960) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.245 to 1.44.246 (#2958) by dependabot[bot]
- bump google.golang.org/api from 0.118.0 to 0.119.0 (#2957) by dependabot[bot]
- bump github.com/klauspost/compress from 1.16.4 to 1.16.5 (#2955) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.244 to 1.44.245 (#2954) by dependabot[bot]
- Development Dependencies:
- bump electron from 24.1.1 to 24.1.2 in /app (#2969) by dependabot[bot]
- bump playwright from 1.32.1 to 1.32.3 in /app (#2970) by dependabot[bot]
- bump @playwright/test from 1.26.1 to 1.32.3 in /app (#2973) by dependabot[bot]
- bump playwright-core from 1.32.1 to 1.32.3 in /app (#2971) by dependabot[bot]
- CI/CD:
- test commit (#2987) by Jarek Kowalski
New in Kopia 0.13.0 RC 2 (Apr 17, 2023)
- Command-Line Interface:
- remove deprecated 'snapshot gc' command (#2862) by Julio Lopez
- General Improvements:
- added unit tests for diff (#2931) by Denis Voytyuk
- added tests for iocopy (#2932) by Denis Voytyuk
- added tests for tlsutil (#2933) by Denis Voytyuk
- fixed remaining checklocks violations (#2939) by Jarek Kowalski
- added unit tests for freepool (#2930) by Denis Voytyuk
- Repository:
- remove metrics.Registry from repo.Repository (#2944) by Jarek Kowalski
- Dependencies:
- bump github.com/aws/aws-sdk-go from 1.44.243 to 1.44.244 (#2943) by dependabot[bot]
- bump github.com/prometheus/client_golang (#2938) by dependabot[bot]
- bump google.golang.org/api from 0.117.0 to 0.118.0 (#2935) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.241 to 1.44.243 (#2937) by dependabot[bot]
- bump github.com/minio/minio-go/v7 from 7.0.51 to 7.0.52 (#2927) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.240 to 1.44.241 (#2926) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.239 to 1.44.240 (#2915) by dependabot[bot]
- Development Dependencies:
- bump concurrently from 7.6.0 to 8.0.1 in /app (#2880) by dependabot[bot]
- bump electron from 23.1.3 to 24.1.1 in /app (#2940) by dependabot[bot]
- bump electron-builder from 24.0.0 to 24.2.0 in /app (#2922) by dependabot[bot]
- fixup mutex for make check-locks (#2920) by Aaron Alpar
- Infrastructure:
- do not remove branches on stale workflow (#2925) by Julio Lopez
- upgrade actions/stale to v8 (#2924) by Julio Lopez
- manage stale issues and pull requests (#2916) by Julio Lopez
- CI/CD:
- don't auto-submit HomeBrew pull requests for testing releases (#2951) by Jarek Kowalski
- trigger CI rebuild after signing key update (#2950) by Jarek Kowalski
New in Kopia 0.13.0 RC 1 (Apr 11, 2023)
- Command-Line Interface:
- remove default behavior for CLI command (#2861) by Julio Lopez
- fixed help formatting broken with recent upgrade to kingpin/v2 (#2868) by Jarek Kowalski
- New Feature ability to read repository status ... (#2799) by Aaron Alpar
- New Feature Make format upgrade command idempotent (#2824) by Ali Dowair
- minor improvements to upgrade CLI usage/documentation (#2686) by Ali Dowair
- do not allow index downgrade by using set-parameter (#2629) by Aaron Alpar
- New Feature Support displaying storage values in base-2 [#2492] (#2502) by atom
- Graphical User Interface:
- update to latest htmlui (#2851) by Jarek Kowalski
- KopiaUI App:
- fixed '' in the tray menu. (#2813) by Jarek Kowalski
- added missing entitlement for M1 build (#2809) by Jarek Kowalski
- General Improvements:
- Fixed ordering in function "isAbs" in ospath (#2850) by Christoph Anderson
- move test-only functions to _test.go file (#2859) by Julio Lopez
- fix-up *Fault with lock (#2855) by Aaron Alpar
- parse stdout and stderr in robustness framework helper (#2779) by Julio Lopez
- correct spelling mistakes (#2684) by Edward Betts
- prevent storage retries on ESTALE errors (#2628) by Shikhar Mall
- introduced generics to reduce boilerplate code (#2527) by Jarek Kowalski
- Repository:
- New Feature live cache eviction of expired BLOBs in persistent LRU content cache (#2879) by Shikhar Mall
- New Feature Add Custom Root CA option for S3 (#2845) by Damien Degois
- fixed connect broken since rclone v1.61.0 (#2853) by YFdyh000
- New Feature added RepositoryWriter.ReplaceManifests (#2747) by Jarek Kowalski
- New Feature added streaming of FindManifests responses (#2733) by Jarek Kowalski
- New Feature increase GRPC maximum message size to 36 MiB (#2713) by budachst
- extract parts repo/content into packages (#2651) by Jarek Kowalski
- New Feature new repository upgrade sub-command to validate indices. (#2561) by Aaron Alpar
- New Feature added internal/metricid package (#2626) by Jarek Kowalski
- New Feature added OnSuccessfulFlush() (#2623) by Jarek Kowalski
- New Feature added before and after callbacks for RepositoryWriter.Flush() (#2622) by Jarek Kowalski
- moved cache protection to separate package (#2621) by Jarek Kowalski
- server testability refactoring (#2612) by Jarek Kowalski
- refactored server repository implementations (#2595) by Jarek Kowalski
- added metrics aggregation, removed gauges (#2594) by Jarek Kowalski
- replaced atomic values with Go 1.19 atomic wrappers (#2590) by Jarek Kowalski
- refactored Repository close logic (#2589) by Jarek Kowalski
- New Feature added Snapshot(bool) option to reset metrics (#2588) by Jarek Kowalski
- refactored Prometheus metrics (#2532) by Jarek Kowalski
- Manifest label comparison when value is empty (#2543) by ashmrtn
- New Feature add more split size options: 128K, 256K, 512K (#2496) by Le Duane
- Server:
- New Feature improved server shutdown and integration tests (#2722) by Jarek Kowalski
- Snapshots:
- New Feature Enable compression for StreamingFile entries (#2835) by ashmrtn
- New Feature Return ReadCloser from StreamingFiles (#2692) by ashmrtn
- cached stream file size (#2690) by ashmrtn
- Partly fix #544 by supporting setuid/setgid/sticky bits when snapshotting, restoring and fuse-mounting. (#2597) by lciti
- New Feature Add labels to snapshot checkpoint manifests (#2548) by ashmrtn
- Dependencies:
- bump google.golang.org/api from 0.116.0 to 0.117.0 (#2914) by dependabot[bot]
- bump github.com/minio/minio-go/v7 from 7.0.50 to 7.0.51 (#2911) by dependabot[bot]
- bump golang.org/x/crypto from 0.7.0 to 0.8.0 (#2904) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.238 to 1.44.239 (#2906) by dependabot[bot]
- bump golang.org/x/oauth2 from 0.6.0 to 0.7.0 (#2905) by dependabot[bot]
- upgrade github.com/hashicorp/golang-lru to 0.6.0 (#2901) by Julio Lopez
- bump github.com/klauspost/compress from 1.16.3 to 1.16.4 (#2896) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.237 to 1.44.238 (#2900) by dependabot[bot]
- bump golang.org/x/net from 0.8.0 to 0.9.0 (#2898) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.236 to 1.44.237 (#2897) by dependabot[bot]
- bump google.golang.org/api from 0.115.0 to 0.116.0 (#2895) by dependabot[bot]
- bump google.golang.org/api from 0.114.0 to 0.115.0 (#2891) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.235 to 1.44.236 (#2892) by dependabot[bot]
- bump golang.org/x/term from 0.6.0 to 0.7.0 (#2893) by dependabot[bot]
- bump golang.org/x/sys from 0.6.0 to 0.7.0 (#2890) by dependabot[bot]
- bump golang.org/x/mod from 0.9.0 to 0.10.0 (#2889) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.234 to 1.44.235 (#2887) by dependabot[bot]
- bump minimist from 1.2.7 to 1.2.8 in /app (#2820) by Julio Lopez
- bump github.com/aws/aws-sdk-go from 1.44.233 to 1.44.234 (#2878) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.232 to 1.44.233 (#2875) by dependabot[bot]
- bump google.golang.org/protobuf from 1.29.1 to 1.30.0 (#2872) by dependabot[bot]
- bump github.com/fatih/color from 1.14.1 to 1.15.0 (#2874) by dependabot[bot]
- bump go.uber.org/multierr from 1.10.0 to 1.11.0 (#2873) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.207 to 1.44.232 (#2871) by dependabot[bot]
- bump go.uber.org/multierr from 1.9.0 to 1.10.0 (#2829) by dependabot[bot]
- bump google.golang.org/grpc from 1.53.0 to 1.54.0 (#2856) by dependabot[bot]
- bump go.opentelemetry.io/otel/exporters/jaeger (#2867) by dependabot[bot]
- bump cloud.google.com/go/storage from 1.29.0 to 1.30.1 (#2866) by dependabot[bot]
- bump github.com/google/fswalker (#2860) by Julio Lopez
- bump actions/setup-go from 3 to 4 (#2841) by dependabot[bot]
- bump github.com/golang/protobuf from 1.5.2 to 1.5.3 (#2828) by dependabot[bot]
- bump github.com/minio/minio-go/v7 from 7.0.49 to 7.0.50 (#2857) by dependabot[bot]
- bump go.opentelemetry.io/otel/sdk from 1.13.0 to 1.14.0 (#2837) by dependabot[bot]
- bump go.opentelemetry.io/otel from 1.13.0 to 1.14.0 (#2826) by dependabot[bot]
- bump github.com/chromedp/chromedp from 0.8.7 to 0.9.1
New in Kopia 0.12.1 (Oct 2, 2022)
- Graphical User Interface:
- icon updates (#2469) by Jarek Kowalski
- Repository:
- fixed frequent repository refreshes for older repositories (#2468) by Jarek Kowalski
- Server:
- Respect manual scheduling policy (#2464) by Ricardo Pescuma Domenecci
- Dependencies:
- bump github.com/aws/aws-sdk-go from 1.44.107 to 1.44.108 (#2462) by dependabot[bot]
- bump github.com/minio/minio-go/v7 from 7.0.38 to 7.0.39 (#2461) by dependabot[bot]
New in Kopia 0.11.3 (Jul 11, 2022)
- Command-Line Interface:
- Removed unnecessary console log timestamps (#2175) by Jarek Kowalski
- New Feature removed unnecessary logs from cli-logs (#2174) by Jarek Kowalski
- Fixed 'cache set' CLI regression due to kingpin change (#2173) by Jarek Kowalski
- Graphical User Interface:
- Fixed filesysystem restores triggered from UI (#2163) by Jarek Kowalski
- Repository:
- Fixed a race where iteration goroutine wasn't closed properly (#2177) by Jarek Kowalski
- Fixed panic when content cache has been disabled (rare) (#2176) by Jarek Kowalski
- Dependencies:
- Bump go.opentelemetry.io/otel/exporters/jaeger (#2168) by dependabot[bot]
- Bump go.opentelemetry.io/otel/sdk from 1.7.0 to 1.8.0 (#2167) by dependabot[bot]
- Bump github.com/aws/aws-sdk-go from 1.44.50 to 1.44.51 (#2166) by dependabot[bot]
- Bump github.com/aws/aws-sdk-go from 1.44.49 to 1.44.50 (#2159) by dependabot[bot]
- Infrastructure:
- Improved support for in-process testing (#2169) by Jarek Kowalski
New in Kopia 0.11.2 (Jul 7, 2022)
- Graphical User Interface:
- upgraded UI (#2157) by Jarek Kowalski
- fixed missing directory browser button in KopiaUI (#2155) by Jarek Kowalski
- "Kopia UI" to "KopiaUI" (#2140) by basldfalksjdf
- Command-Line Interface:
- fixed snapshot sizes in the snapshot list (#2148) by Jarek Kowalski
- Snapshots:
- panic: unaligned 64-bit atomic operation (#2151) by Jarek Kowalski
- Dependencies:
- bump github.com/aws/aws-sdk-go from 1.44.48 to 1.44.49 (#2156) by dependabot[bot]
- bump github.com/minio/minio-go/v7 from 7.0.30 to 7.0.31 (#2154) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.44.47 to 1.44.48 (#2146) by dependabot[bot]
New in Kopia 0.11.1 (Jul 5, 2022)
- Graphical User Interface:
- Fixed scheduling of snapshots with Ignoring Identical Snapshots (#2141) by Jarek Kowalski
New in Kopia 0.10.7 (Apr 7, 2022)
- Command-Line Interface:
- New Feature show storage capacity in repo status
- New Feature improved kopia benchmark commands
- New Feature add --json output to 'repo status' command
- Fixing ignoring lines starting with '#' in 'policy edit'
- Fixed and unified help text for policy commands
- General Improvements:
- Add a test method to perform repository format version upgrade
- Repository:
- Extracted content index building and parsing into repo/content/index
- New Feature automatically compress all kopia internal metadata using zstd-fastest
- Fixed accidental cache directory location change during refactoring
- Unified data and metadata cache implementations
- Fixed 'unable to get content data and info'
- Major server code refactoring
- New Feature ensure we don't run parallel fetches for the same blob IDs
- Switched from opencensus to directly exporting prometheus metrics
- Minor refactor for common code path in helper
- Streamlined internal content cache API
- Moved index blob cache to separate directory
- New Feature added 'hint' to Prefetch methods.
- Fix deduplication when snapshotting identical files in parallel
- Server:
- Fixed server startup race condition
- Snapshots:
- New Feature support for controlling upload parallelism via policies
- New Feature support restoring sparse files
- Fixed snapshotting of mounted VSS snapshot roots
- Storage Providers:
- New Feature Implement API to get Storage free space
- Dependencies:
- Bump google.golang.org/api from 0.73.0 to 0.74.0
- Bump github.com/aws/aws-sdk-go from 1.43.26 to 1.43.31
- Bump github.com/zalando/go-keyring from 0.1.1 to 0.2.1
- Bump github.com/sanity-io/litter from 1.5.2 to 1.5.4
- Bump github.com/zeebo/blake3 from 0.2.2 to 0.2.3
- Bump google.golang.org/protobuf from 1.27.1 to 1.28.0
- Bump github.com/chromedp/chromedp from 0.7.8 to 0.8.0
- Bump github.com/aws/aws-sdk-go from 1.43.9 to 1.43.26
- Bump github.com/stretchr/testify from 1.7.0 to 1.7.1
- Bump github.com/golang-jwt/jwt/v4 from 4.3.0 to 4.4.1
- Bump github.com/klauspost/compress from 1.14.4 to 1.15.1
- Bump google.golang.org/api from 0.70.0 to 0.73.0
- Bump plist from 3.0.4 to 3.0.5 in /app
- Upgrade minimist to 1.2.6
- Development Dependencies:
- Bump electron from 17.2.0 to 18.0.1 in /app
- Bump electron-notarize from 1.1.1 to 1.2.1 in /app
- Bump electron from 17.1.0 to 17.2.0 in /app
- CI/CD:
- Fixed TestThrottler flake
- Fixed flaky TestServerControl
- Added checklocks static analyzer
- Upgraded linter to 1.45
- Skip non-deterministic tests when computing code coverage
New in Kopia 0.10.6 (Mar 7, 2022)
- Graphical User Interface:
- New Feature display snapshot description in snapshot list (#1806) by Jarek Kowalski
- Command-Line Interface:
- New Feature improved formatting of 'policy show' outputs (#1767) by Jarek Kowalski
- New Feature added snapshot list flags: --storage-stats and --reverse (#1766) by Jarek Kowalski
- KopiaUI App:
- New Feature support both left and right click to open tray menu (#1805) by Jarek Kowalski
- New Feature provide feedback for 'Check For Updates Now' (#1804) by Jarek Kowalski
- General Improvements:
- New Feature Upgrade lock retries & monitoring during open & write sessions (#1758) by Shikhar Mall
- Repository:
- New Feature removed noisy log output (#1803) by Jarek Kowalski
- better context cancelation handling (#1802) by Jarek Kowalski
- New Feature added PrefetchObjects() API (#1779) by Jarek Kowalski
- added context to beforeop callbacks (#1800) by Jarek Kowalski
- fixed 'context canceled' regression (#1799) by Jarek Kowalski
- expose ContentInfo() on repo.Repository (#1765) by Jarek Kowalski
- New Feature repository writer upgrade lock apis (#1742) by Shikhar Mall
- Snapshots:
- refactored TreeWalker to use workshare (#1762) by Jarek Kowalski
- New Feature big upload performance improvements by parallelizing directory traversal (#1752) by Jarek Kowalski
- Storage Providers:
- Instructions for how to use the Google Drive backend. (#1760) by xkxx
- New Feature add Google Drive support (#1731) by Jarek Kowalski
- Dependencies:
- bump go.uber.org/zap from 1.20.0 to 1.21.0 (#1795) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.42.44 to 1.43.9 (#1785) by dependabot[bot]
- bump google.golang.org/api from 0.69.0 to 0.70.0 (#1791) by dependabot[bot]
- bump github.com/minio/minio-go/v7 from 7.0.21 to 7.0.23 (#1794) by dependabot[bot]
- bump github.com/chromedp/chromedp from 0.7.6 to 0.7.8 (#1793) by dependabot[bot]
- bump github.com/klauspost/compress from 1.14.2 to 1.14.4 (#1792) by dependabot[bot]
- bump nanoid from 3.1.30 to 3.3.1 in /app (#1790) by dependabot[bot]
- bump electron-updater from 4.6.1 to 4.6.5 in /app (#1781) by dependabot[bot]
- bump go.uber.org/multierr from 1.7.0 to 1.8.0 (#1783) by dependabot[bot]
- bump actions/checkout from 2 to 3 (#1782) by dependabot[bot]
- bump github.com/golang-jwt/jwt/v4 from 4.2.0 to 4.3.0 (#1787) by dependabot[bot]
- bump electron-log from 4.4.5 to 4.4.6 in /app (#1788) by dependabot[bot]
- bump cloud.google.com/go/storage from 1.19.0 to 1.21.0 (#1789) by dependabot[bot]
- Development Dependencies:
- bump electron from 17.0.0 to 17.1.0 in /app (#1784) by dependabot[bot]
- Infrastructure:
- upgraded linter to 1.44.2 (#1768) by Jarek Kowalski
- CI/CD:
- New Feature automatically bump homebrew version on tag (#1807) by Jarek Kowalski
- remove changelog type filter (#1761) by Jarek Kowalski
New in Kopia 0.10.5 (Feb 12, 2022)
- Graphical User Interface:
- New Feature support for editing pins and description (#1748) by Jarek Kowalski
- New Feature support for defining snapshot and folder actions (#1746) by Jarek Kowalski
- fix parsing of times of day between 00:00 and 00:59 (#1736) by Jarek Kowalski
- New Feature support for snapshot and source deletion (#1730) by Jarek Kowalski
- General Improvements:
- New Feature Upgrade Lock intent definitions (#1727) by Shikhar Mall
- Misc nits (#1729) by Julio Lopez
- Snapshots:
- Treat empty retention policy as retaining ALL, not NONE (#1733) by Jarek Kowalski
- Dependencies:
- Bump follow-redirects from 1.14.6 to 1.14.8 in /app (#1745) by dependabot[bot]
- Update storage/azblob to v0.3.0 (#1743) by Jarek Kowalski
- CI/CD:
- Auto-cancel previous workflows on the same branch (#1747) by Jarek Kowalski
New in Kopia 0.10.4 (Feb 3, 2022)
- Command-Line Interface:
- New Feature allow changing retention parameters from CLI (#1680) by Shikhar Mall
- An in-memory storage mock setup for CLI tests (#1697) by Shikhar Mall
- Storage Providers:
- New Feature treat token expiration errors as non-retryable (#1675) by Onkar Bhat
- fixed rclone connection in KopiaUI (#1712) (#1713) by Jarek Kowalski
- Dependencies:
- bump github.com/minio/minio-go/v7 from 7.0.20 to 7.0.21 (#1715) by dependabot[bot]
- bump github.com/prometheus/client_golang (#1716) by dependabot[bot]
- bump cloud.google.com/go/storage from 1.18.2 to 1.19.0 (#1714) by dependabot[bot]
- bump github.com/google/go-cmp from 0.5.6 to 0.5.7 (#1708) by dependabot[bot]
- bump github.com/klauspost/compress from 1.13.6 to 1.14.2 (#1704) by dependabot[bot]
- bump github.com/sanity-io/litter from 1.5.1 to 1.5.2 (#1709) by dependabot[bot]
- bump electron-log from 4.4.4 to 4.4.5 in /app (#1700) by dependabot[bot]
- bump github.com/zeebo/blake3 from 0.2.1 to 0.2.2 (#1703) by dependabot[bot]
- bump google.golang.org/grpc from 1.43.0 to 1.44.0 (#1710) by dependabot[bot]
- bump google.golang.org/api from 0.63.0 to 0.66.0 (#1702) by dependabot[bot]
- bump github.com/aws/aws-sdk-go from 1.42.25 to 1.42.44 (#1706) by dependabot[bot]
- bump go.uber.org/zap from 1.19.1 to 1.20.0 (#1707) by dependabot[bot]
- Development Dependencies:
- bump electron from 16.0.5 to 17.0.0 in /app (#1705) by dependabot[bot]
- bump electron-builder from 22.14.11 to 22.14.13 in /app (#1699) by dependabot[bot]
- bump concurrently from 6.5.1 to 7.0.0 in /app (#1701) by dependabot[bot]
- CI/CD:
- add rclone binaries to the list of artifacts (#1719) by Jarek Kowalski
- added rclone 1.57.0 to Docker container images (#1718) by Jarek Kowalski
New in Kopia 0.10.3 (Jan 31, 2022)
- Graphical User Interface:
- Fixed unwanted refresh of directory listing (#1694) by Jarek Kowalski
- CI/CD:
- Cleaned up changelog output (#1695) by Jarek Kowalski
New in Kopia 0.10.2 (Jan 30, 2022)
- KopiaUI App:
- New Feature connect to repository asynchronously on startup
- Repository:
- New Feature persistence for blob-retention configuration
- Server:
- sleep 30m after failed maintenance
- Storage Providers:
- fixed DoNotRecreate and tests for gcs
- New Feature expand PutBlob API to allow for idempotent puts
- New Feature improved sharded directory creation
- Dependencies:
- upgrade go-fuse to support /bin/fusermount3
- Testing:
- fixed robustness tests
- Linter:
- enabled gochecknoglobals and tagged existing globals
- CI/CD:
- New Feature automatically publish official releases
- improve changelog readability
- fixed linter issue, do not ignore in workflow
- generate changelog as part of the build
- auto-generate change log
New in Kopia 0.10.1 (Jan 14, 2022)
- fix(ci): fix regression when publishing APT repository by @jkowalski in #1658
- fix(ui): fixed HTTP 400 response when repository is not connected by @jkowalski in #1659
New in Kopia 0.10.0 (Jan 14, 2022)
- What's Changed:
- logging: log maximum concurrency level when talking to backend storage by @jkowalski in #1629
- kopiaui: added menu option prompting users to manually upgrade when auto-update fails by @jkowalski in #1655
- ui: embed version info in index.html by @jkowalski in #1656
- Dependencies:
- build(deps-dev): bump electron-builder from 22.14.6 to 22.14.11 in /app by @dependabot in #1640
- build(deps-dev): bump concurrently from 6.5.0 to 6.5.1 in /app by @dependabot in #1638
- build(deps): bump electron-log from 4.4.3 to 4.4.4 in /app by @dependabot in #1639
- build(deps): bump github.com/minio/minio-go/v7 from 7.0.18 to 7.0.20 by @dependabot in #1642
- build(deps): bump github.com/aws/aws-sdk-go from 1.42.23 to 1.42.25 by @dependabot in #1641
- Nits:
- small error on domain name example at the end. by @icepic in #1627
- nit: fixed benign test data race by @jkowalski in #1635
- server: moved serving of static files to internal/server package by @jkowalski in #1637
- testing: fixed test flake in TestSnapshotCounters by @jkowalski in #1645
- chore(infra): enforce structural commit messages and PR titles by @jkowalski in #1646
New in Kopia 0.9.8 (Dec 26, 2021)
- blob/s3: unconditionally compute and send the MD5 header by @julio-lopez in #1569
- repo: removed old, unused field used in the past for non-encrypted repositories by @jkowalski in #1573
- cli: reduce number of index blob writes in 'snapshot create --all' by @jkowalski in #1574
- test coverage improvements for filesystem provider by @jkowalski in #1576
- More coverage improvements by @jkowalski in #1577
- localfs: adjust build flags to support more archs by @daniel-eys in #1183
- blobtesting: refactored fault injection into fluent API by @jkowalski in #1578
- content: added direct code coverage for mergedIndex by @jkowalski in #1580
- scrubber: added unit tests by @jkowalski in #1581
- cache: improved test coverage by @jkowalski in #1582
- content: additional tests form encryptedBlobMgr by @jkowalski in #1583
- content: additional data & metadata cache test cases by @jkowalski in #1584
- testing: do not run randomized tests in code coverage mode by @jkowalski in #1585
- epoch: changed how very slow index writes are handled by @jkowalski in #1587
- azure: migrated provider to newer client library, remove gocloud.dev dependency by @jkowalski in #1589
- b2: fixed regression when uploading zero-length blobs introduced by #1589 by @jkowalski in #1593
- test: run provider tests when pushing to test/providers branch by @julio-lopez in #1594
- versioned map mock storage for blob-retention unit-testing by @Shrekster in #1590
- Fix typo in policy help documentation by @strager in #1597
- htmlui: moved HTML UI to separate repository by @jkowalski in #1600
- build(deps-dev): bump concurrently from 6.4.0 to 6.5.0 in /app by @dependabot in #1601
- blob: replaced blob.Storage.SetTime() method with blob.PutOptions.SetTime by @jkowalski in #1595
- build(deps-dev): bump react-scripts from 4.0.3 to 5.0.0 in /app by @dependabot in #1603
- build(deps): bump electron-log from 4.4.1 to 4.4.3 in /app by @dependabot in #1604
- upgrade htmlui by @jkowalski in #1610
- build(deps-dev): bump electron from 16.0.3 to 16.0.5 in /app by @dependabot in #1602
- build(deps): bump github.com/aws/aws-sdk-go from 1.42.16 to 1.42.23 by @dependabot in #1606
- build(deps): bump github.com/golang-jwt/jwt/v4 from 4.1.0 to 4.2.0 by @dependabot in #1609
- build(deps): bump github.com/minio/minio-go/v7 from 7.0.16 to 7.0.18 by @dependabot in #1605
- build(deps): bump google.golang.org/grpc from 1.42.0 to 1.43.0 by @dependabot in #1608
- build(deps): bump google.golang.org/api from 0.60.0 to 0.63.0 by @dependabot in #1607
- Allow building without UI by @julio-lopez in #1614
- s3/test: refactor retention tests by @julio-lopez in #1615
- s3/test: fix minio bucket creation in tests by @julio-lopez in #1617
- maintenance: moved costly epoch manager cleanup to maintenance by @jkowalski in #1616
- s3/test: cleanup blob versions by @julio-lopez in #1618
- ui: added preliminary dark mode support (the choice is not persistent yet) by @jkowalski in #1619
- ui: apply dark mode default and persist user choice by @jkowalski in #1621
- ui: cleaned up task details page by @jkowalski in #1623
New in Kopia 0.9.7 (Dec 6, 2021)
- What's Changed:
- ui: revamped policy editor + various UI cleanups. by @jkowalski in #1537
- restore: use non-atomic writes by @0x53A in #1534
- policy: revamped policy merge mechanism by @jkowalski in #1538
- ui: Policy Editor - show effective value and definition point for policy fields by @jkowalski in #1545
- b2: fixed error handling for GetMetadata() by @jkowalski in #1563
- cache: prevent metadata cache thrashing if working set exceeds max defined size by @jkowalski in #1557
- logging: added log rotation and improved predictability of log sweep by @jkowalski in #1562
- ui: major improvements to new snapshot flow by @jkowalski in #1565
- ui: added editor for snapshot times of day by @jkowalski in #1566
- ui: add CLI equivalent button to most pages by @jkowalski in #1568
- ui/cli: merge retention tags in snapshot lists by @jkowalski in #1567
- Under-The Hood Changes:
- build(deps): bump react-bootstrap from 2.0.2 to 2.0.3 in /htmlui by @dependabot in #1556
- build(deps): bump github.com/minio/minio-go/v7 from 7.0.15 to 7.0.16 by @dependabot in #1553
- build(deps-dev): bump concurrently from 6.3.0 to 6.4.0 in /app by @dependabot in #1548
- build(deps): bump github.com/mattn/go-colorable from 0.1.11 to 0.1.12 by @dependabot in #1551
- build(deps-dev): bump postcss from 8.3.11 to 8.4.4 in /htmlui by @dependabot in #1546
- build(deps-dev): bump electron from 15.3.0 to 16.0.3 in /app by @dependabot in #1547
- build(deps-dev): bump @testing-library/jest-dom from 5.15.0 to 5.15.1 in /htmlui by @dependabot in #1555
- build(deps): bump github.com/aws/aws-sdk-go from 1.41.16 to 1.42.16 by @dependabot in #1552
- build(deps-dev): bump electron-builder from 22.13.1 to 22.14.5 in /app by @dependabot in #1550
- build(deps): bump electron-updater from 4.3.9 to 4.6.1 in /app by @dependabot in #1549
- htmlui: upgraded react-bootstrap to 2.x by @jkowalski in #1532
- Documentation Updates:
- Replace google with gcs in examples by @kido5217 in #1533
- Add doc for sharding by @CrendKing in #1536
- New Contributors:
- @kido5217 made their first contribution in #1533
- @0x53A made their first contribution in #1534
New in Kopia 0.9.6 (Nov 21, 2021)
- ci: dependency upgrades by @jkowalski in #1504
- upload: avoid reporting ignored entries twice by @jkowalski in #1506
- Add OpenBSD instructions by @icepic in #1508
- Upgraded linter to 1.43.0 by @jkowalski in #1505
- blob: added throttling wrapper by @jkowalski in #1510
- storage api put-blob retention options by @Shrekster in #1511
- blob: changed default shards from {3,3} to {1,3} by @jkowalski in #1513
- throtting: implemented a Throttler based on token bucket and configur… by @jkowalski in #1512
- ui: fixed 'Do not validate TLS certificate' caused by typo by @jkowalski in #1523
- Change Mbit/s units to MB/s (base-10) by @CrendKing in #1522
- ui: Added test to verify #1057 by @jkowalski in #1526
- Cleanup cli2md to fix issue #1440 by @CrendKing in #1525
- cli: added 'snapshot pin' command by @jkowalski in #1528
- cli: fixed 'snapshot list --json --max-results' by @jkowalski in #1529
New in Kopia 0.9.5 (Nov 7, 2021)
- Fixed homebrew upgrade/cleanup warning by @t-rutter in #1447
- Use blob.OutputBuffer in blob.Reader interface instead of internal gather.WriteBuffer by @dzaninovic in #1452
- Added support for logging policies by @jkowalski in #1472
- Performance: optimized restore performance for webdav/rclone by @jkowalski in #1491
- Maintenance: improved scheduling by @jkowalski in #1493
- Epoch: eliminate redundant ListBlobs() calls in epoch.Manager by @jkowalski in #1492
- Server: fixed maintenance in newly-created repo by @jkowalski in #1494
- Server: ensure all HTTP requests are processed in a detached context by @jkowalski in #1495
- Maintenance: send logs to content log as well by @jkowalski in #1496
- Ci: build arm64 linux UI by @jkowalski in #1482
- Ci: also build armv7l version of KopiaUI by @jkowalski in #1483
New in Kopia 0.9.4 (Oct 23, 2021)
- Changelog:
- clock: discard monotonic clock component in clock.Now() (#1437)
- cli: when migrating snapshot honor destination policies (#1433)
- cli: clarified usage of 'policy set --snapshot-time` (#1430)
- sftp: support for password authentication (#1425)
- ui: fixed the refresh button in source list (#1428)
- site: rewrote upgrade safety notice (#1424)
- cli: 'repo sync-to' will compare only unique ID not the entire format (#1423)
- migrate: pick latest instead of oldest snapshot with --latest-only (#1420)
- epoch: do not fail on moving deletion watermark backwards (#1422)
- server: fixed next snapshot time computation (#1418)
- epoch: additional logging to get to the bottom of #1402 (#1417)
- cli: benchmark compressor memory usage (#1416)
New in Kopia 0.9.3 (Oct 18, 2021)
- upload: fix ignoring files after action redirects snapshot directory (#1409)
- epoch: increase epoch time and cleanup safety margin (#1407)
- epoch: when doing background compaction and cleanup work, ignore cancelation signal (#1406)
- cli: added 'index inspect --parallel' flag (#1404)
- mount: fixed name escaping and added disambiguation (#1403)
- ui: fixed snapshotting UNC roots (#1401)
- actions: expose KOPIA_ACTION and KOPIA_VERSION environment variables (#1398)
- cli: added 'snapshot delete --all-snapshots-for-source' (#1397)
- localfs: added ReadDir() benchmark (#1396)
- site: mark v0.9 as stable, updated release docs (#1393)
- ci: switched code coverage tool to use GH action (#1391)
New in Kopia 0.9.2 (Oct 15, 2021)
- logging: fixed logging regression in v0.9.1- #1388
- kopia-ui: improved auto-update experience - #1390
New in Kopia 0.9.1 (Oct 14, 2021)
- testing: added tests for filesystem --flat option (#1384)
- retry: abort retry loop on context errors (#1383)
- rclone: pass directory shards to webdav client (#1382)
- logging: fixed console timestamps to be local time, more nits (#1381)
- nit: minor log output cleanups (#1380)
- logging: switched from go-logging to zap (#1376)
- Add doc for Amazon S3 .storageconfig (#1375)
- add unicode filename test, add env switches for long filenames/unicode filenames, update workflow file to include env variables (#1371)
- logging: added memoization of Logger instances per context (#1369)
New in Kopia 0.9.0 (Oct 8, 2021)
- Providers:
- azure: added support for using SAS Tokens instead of storage keys (#1093)
- rclone: wait for rclone transfers to finish when closing (#1201)
- sftp: added automatic reconnect and connection pooling (#1195)
- sftp: ensure key file and known hosts are absolute paths (#1092)
- s3: point-in-time support for s3 versioned stores (#1259)
- webdav & and rclone support for atomic writes (#1206)
- webdav: disable server-side compression (#1100)
- Repository features:
- new index format supporting internal compression and does not require quick maintenance for performance reasons
- splitter: major performance improvements (#1251)
- implemented epoch-based index manager (#1174)
- internal logging - low-level logs are now stored in the repository.
- support for content-level compression (#1076)
- added 'lz4' and 'deflate' methods (#1077)
- added sharding of indexes on write (#1176)
- repository password change support (#1197)
- limit the duration of kopia.repository caching to 15 minutes (#1196)
- encryption: removed old, non-authenticated encryption methods (#979)
- big performance improvement for WriteContent with repo server (#1182)
- disabled quick maintenance for new repositories
- mount: fixed mounting of directories w/symlinks over WebDAV (#1358)
- localfs: fixed support for snapshotting UNC path root (serverpath) (#1362)
- CLI:
- 'kopia content verify' performance improvements (#1120)
- add a flag to not shallowrestore small files (#1126)
- added 'kopia repo set-parameters' to override mutable parameters (#1148)
- added 'kopia repository validate-provider` (#1205)
- additional 'index inspect' flags (#1231)
- changed 'kopia snapshot verify --verify-files-percent' to float (#1210)
- fixed ETA estimation of 'snapshot verify' (#1213)
- force-hash: Allow fractional percentages (#1209)
- include parameters in maintenance info JSON output (#981)
- optimized 'kopia index recover' by leveraging partial read and parallelism (#1094)
- support for shallow restore (#725)
- tagging of kopia snapshots and listing of snapshots by tag (#1030)
- logging: switched file log timestamps to be UTC by default, otherwise it's hard to correlate logs (#1240)
- only read first 128 MiB of provided file for compression benchmark (#1317)
- linux: read KOPIA_USE_KEYRING env (#1337)
- Policy Changes:
- Fix weekly retention labels not being assigned correctly (#1211)
- policy: path validation (#1006)
- UI Changes:
- do not attempt running maintenance if the current user is not the maintenance owner, to avoid producing error in the Tasks tab (#1010)
- fixed Estimate not honoring the defined policies (#1002)
- for read-only repositories start a read-only source manager (#1009)
- open .gitignore help in a new window (#1016)
- show a spinner when saving/deleting policy (#1018)
- fixed directory selector and browse buttons (#1310)
- fixed showing empty result pages (#1306)
- always show default repository at the top of the list (#1349)
- add simple page size selector (#1355)
- Security:
- security: switched to github.com/golang-jwt/jwt/v4 to fix upstream security issue (#1235)
- Notable internal Changes:
- automatically refresh indexes on read instead of relying on RefreshPeriodically() loop (#1243)
- ensure that we always consistently pick content.Info amongst entries with identical time and deleted flag (#1239)
- refactored content.Info to be an interface and switched index parsing to be lazy (#1008)
- refactored own writes cache and list cache into blob.Storage wrappers (#1133)
- few subtle threading bugs uncovered by stress test and rewrote the test to be model-based (#1157)
- reworked memory management to avoid allocating large chunks of memory for most buffers
- ci: Allow to compile Kopia cli on OpenBSD (#983)
- upgrade golang to 1.17
- testing: Refactored most of the CLI tests to run in-process as opposed to using sub-processes (#1059)
New in Kopia 0.9.0 RC 4 (Oct 4, 2021)
- Support setting AWS S3 storage class for all types of blobs (#1335)
- build(deps): bump @testing-library/react in /htmlui (#1327)
- Remove unnecessary subdirectory from Windows archive (#1352)
- build(deps): bump postcss from 8.3.6 to 8.3.8 in /htmlui (#1323)
- add conditional to KopiaUI makefile to build arm64 and x64 versions (#1351)
- kopia-ui: always show default repository at the top of the list (#1349)
- localfs: reduce memory usage when scanning short directories (#1343)
- build(deps-dev): bump electron from 14.0.0 to 15.1.0 in /app (#1328)
- go.mod tidy (#1347)
- build(deps-dev): bump asar from 3.0.3 to 3.1.0 in /app (#1326)
- build(deps): bump bootstrap from 5.1.0 to 5.1.1 in /htmlui (#1325)
- go.mod: upgrade minor versions of dependencies (#1340)
- Add example of Windows shadow copy to Actions doc (#1319)
- cachefs: make cache read O(1) instead of O(n) (#1341)
- deps: upgrade to github.com/golang-jwt/jwt/[email protected] and fixed linter (#1342)
- build(deps-dev): bump concurrently from 6.2.1 to 6.2.2 in /app (#1324)
- build(deps): bump axios from 0.21.4 to 0.22.0 in /htmlui (#1329)
- build(deps): bump github.com/fatih/color from 1.12.0 to 1.13.0 (#1334)
- upload: memory usage and speed improvements (#1336)
- cli: linux: read KOPIA_USE_KEYRING env (#1337)
- minor spelling and grammar fixes (#1318)
New in Kopia 0.9.0 RC 3 (Sep 27, 2021)
- cli: only read first 128 MiB of provided file for compression benchmark (#1317)
- performance: allow reusing of object writer buffers (#1315)
- ui: fixed directory selector and browse buttons (#1310)
- ui: fixed showing empty result pages (#1306)
New in Kopia 0.9.0 RC 2 (Sep 20, 2021)
- cli: fixed error message when trying 'repo sync-to from-config' and the config file is connected to the repository server (#1304)
- blob: support for custom blob store sharding (#1299)
- cache: when cache validation fails, properly reset output buffer (#1295)
- linter: upgrade to 1.42.1 (#1292)
- fixed minor buffer leak from internal logger, added allocation tracker (#1290)
- cli: added support for passing bare filename for --config-file parameter (#1293)
- site: 0.9 release notes (#1291)
New in Kopia 0.8.3 (Apr 22, 2021)
- A hotfix release that addresses two issues in CLI and Kopia UI. It is a recommended upgrade for all users.:
- #1001 [Bug] [Kopia-UI] Policies not detected
- #1003 [Bug][Kopia-UI]Trailing slash in path leads to incorrect behaviour.
New in Kopia 0.8.2 (Apr 17, 2021)
- New Features:
- Full Maintenance is now enabled by default with an interval of 24 hours.
- Added support for configurable Maintenance Safety to speed up cleanups in certain cases.
- Added support for --json flag to certain CLI commands which will produce machine-readable snasphot, policy, manifest, blob and content responses.
- Added support for scanning only one filesystem via files policy
- Major improvements to ignore logic to better follow .gitignore rules.
- Added support for Actions which allow running custom scripts before and after a directory is snapshotted with lots of useful applications.
- Added support for ACLs which allow defining custom access control rules for repository server clients.
- Docker Images are now available
- Support for write sessions, which ensure garbage collection does not prematurely delete blobs that are in the middle of being uploaded by intermittently-connected clients (e.g. laptops that go to sleep often and can't checkpoint frequently enough)
- Added support for stdin snapshots.
- Added support for user authentication using user profiles stored in the repository (#809)
- Brand new streaming GRPC protocol for talking to kopia repository server to improve performance and ensure session stickyness, which allows sharding and load-balancing.
- Reworked error handling when taking snapshots to not fail fast (except when explicitly requested).
- Switched to new FUSE implementation which supports MacFUSE 4.x
- Added WebDAV-based mounting alternative for platforms that don't have working FUSE mechanism.
- Streamlined flags for caching and improved cache sweep.
- Major performance improvements when performing lots of index lookups in heavily-fragmented indexes (speeds up snapshot verification by orders of magnitude)
- Passthrough options to allow other-users access and mounting in empty directory.
- Don't run maintenance as part of read-only actions.
- Implemented caching for contents downloaded from Kopia Repository Server
- Reduced latency for kopia repository server uploads by performing existence checks
- Improvements to snapshot verify to make output easier to understand.
- compress: upgrade klauspost/compress to v1.11.12 (#888)
- Terminal password prompt - switched to golang.org/x/term (#898)
- New KopiaUI features:
- Added new tab called Tasks which shows the status, progress, logs, and statistics of current and past long-running tasks (restore, estimate, snapshot, maintenance).
- Streamlined snapshot creation flow to include estimation and initial policy definition.
- Provider improvements:
- S3 client - upgraded minio to v7
- rclone - testing improvements
- S3, GCS, Azure, B2 - rewrote retry logic to retry on all but known fatal errors.
- S3 - support for buckets configured with object locking
- Infrastructure:
- Build for Apple Silicon (M1) is now available. The CLI is now dual-architecture
- and runs natively on both amd64 and arm64.
- Switched CI/CD infrastructure to GitHub actions with two self-hosted runners for ARMHF and ARM64
- Makefile: various tricks to make PR builds faster
- S3: run tests against private minio deployment in docker (#894)
- SFTP: fixed retry logic
- RClone: improved error handling logic
- Compatibility Information:
- v0.8 release is compatible with repositories created with v0.7, v0.6 and v0.5.
- While older clients can read repositories created with v0.8 it's recommended to use the latest version at earliest convenience.
- Breaking Changes:
- When starting kopia server with --htpasswd-file, the UI access is now only granted to a user named kopia (if it exists in the password file). If such user does not exist in the password file, --server-username flag may be specified to indicate which of the users listed in the password file gets UI access (there can be only one). See #880 for more information.
- Added explicit --insecure flag to kopia server start to prevent accidentally starting non-secured
- servers for development. Starting the server without TLS or without authentication now requires this flag.
- Several low-level maintenance tools have been changed to support individual parameters with --safety=full|none flag instead of specific parameters which were unsafe.
New in Kopia 0.8.1 (Apr 12, 2021)
- New Features:
- Full Maintenance is now enabled by default with an interval of 24 hours.
- Added support for configurable Maintenance Safety to speed up cleanups in certain cases.
- Added support for --json flag to certain CLI commands which will produce machine-readable snasphot, policy, manifest, blob and content responses.
- Added support for scanning only one filesystem via files policy
- Major improvements to ignore logic to better follow .gitignore rules.
- Added support for Actions which allow running custom scripts before and after a directory is snapshotted with lots of useful applications.
- Added support for ACLs which allow defining custom access control rules for repository server clients.
- Docker Images are now available
- Support for write sessions, which ensure garbage collection does not prematurely delete blobs that are in the middle of being uploaded by intermittently-connected clients (e.g. laptops that go to sleep often and can't checkpoint frequently enough)
- Added support for stdin snapshots.
- Added support for user authentication using user profiles stored in the repository (#809)
- Brand new streaming GRPC protocol for talking to kopia repository server to improve performance and ensure session stickyness, which allows sharding and load-balancing.
- Reworked error handling when taking snapshots to not fail fast (except when explicitly requested).
- Switched to new FUSE implementation which supports MacFUSE 4.x
- Added WebDAV-based mounting alternative for platforms that don't have working FUSE mechanism.
- Streamlined flags for caching and improved cache sweep.
- Major performance improvements when performing lots of index lookups in heavily-fragmented indexes (speeds up snapshot verification by orders of magnitude)
- Passthrough options to allow other-users access and mounting in empty directory.
- Don't run maintenance as part of read-only actions.
- Implemented caching for contents downloaded from Kopia Repository Server
- Reduced latency for kopia repository server uploads by performing existence checks
- Improvements to snapshot verify to make output easier to understand.
- compress: upgrade klauspost/compress to v1.11.12 (#888)
- Terminal password prompt - switched to golang.org/x/term (#898)
- New KopiaUI features:
- Added new tab called Tasks which shows the status, progress, logs, and statistics of current and past long-running tasks (restore, estimate, snapshot, maintenance).
- Streamlined snapshot creation flow to include estimation and initial policy definition.
- Provider improvements:
- S3 client - upgraded minio to v7
- rclone - testing improvements
- S3, GCS, Azure, B2 - rewrote retry logic to retry on all but known fatal errors.
- S3 - support for buckets configured with object locking
- Infrastructure:
- Build for Apple Silicon (M1) is now available. The CLI is now dual-architecture
- and runs natively on both amd64 and arm64.
- Switched CI/CD infrastructure to GitHub actions with two self-hosted runners for ARMHF and ARM64
- Makefile: various tricks to make PR builds faster
- S3: run tests against private minio deployment in docker (#894)
- SFTP: fixed retry logic
- RClone: improved error handling logic
- Compatibility Information:
- v0.8 release is compatible with repositories created with v0.7, v0.6 and v0.5.
- While older clients can read repositories created with v0.8 it's recommended to use the latest version at earliest convenience.
- Breaking Changes:
- When starting kopia server with --htpasswd-file, the UI access is now only granted to a user named kopia (if it exists in the password file). If such user does not exist in the password file, --server-username flag may be specified to indicate which of the users listed in the password file gets UI access (there can be only one). See #880 for more information.
- Added explicit --insecure flag to kopia server start to prevent accidentally starting non-secured
- servers for development. Starting the server without TLS or without authentication now requires this flag.
- Several low-level maintenance tools have been changed to support individual parameters with --safety=full|none flag instead of specific parameters which were unsafe.
New in Kopia 0.8.0 Beta 4 (Apr 3, 2021)
- Support for --safety=none when running maintenance which removes all time delays and makes maintenance cleanups very quick
- Support for --json output for several CLI commands
- S3: support for buckets configured with object locking
- Misc provider cleanups and stability updates
New in Kopia 0.8.0 Beta 3 (Mar 21, 2021)
- site: installation instructions and updated release notes (#901)
- Rollback github.com/efarrer/iothrottler to v0.0.1 (#900)
- ci: install signing tools just before building the binary with UI
- ci/cd: refactored how GH releases are published (#867)
- Terminal password prompt - switched to golang.org/x/term (#898)
- Misc cleanups (#899)
- Dockerfile: switched to distroless, specified defaults environment variables for containerized kopia (#897)
- docker: publish docker images to docker hub (#896)
- Added repository user authorization support + server flag refactoring + refresh (#890)
- testing: ensure tests are releasing all buffer pools to reduce memory usage, we had huge leaks (#895)
- s3: run tests against private minio deployment in docker (#894)
- Add context to robustness engine interfaces (#893)
- Revert "ci: run build every 4 hours to get more test samples (temporary)"
- Revert "Revert "compress: upgrade klauspost/compress to v1.11.12 (#888)" (#891)"
- compression: added benchmark to measure impact of library upgrades (#892)
- eaf14a5 Path protection between robustness engine FileWriter and Snapshotter (#865)
- Revert "compress: upgrade klauspost/compress to v1.11.12 (#888)" (#891)
- ci: run build every 4 hours to get more test samples (temporary)
- Add Arch-specific installation instructions (#889)
- compress: upgrade klauspost/compress to v1.11.12 (#888)
- ci: prevent locking macos keychain if the build takes more than 5 minutes
New in Kopia 0.8.0 Beta 2 (Mar 15, 2021)
- Ui: show the current username@hostname as it is connecting to repository (#886)
- Server: ensure we reject access to the UI static files for users other than the UI user (#884)
- Fixed error flicker - replaced fancy error Popover which flickers with a simple tooltip. (#883)
- Htmlui: urlencode path components (#882)
- Site: updated release notes after v0.8.0-beta1 release, added breaking changes (#881)
New in Kopia 0.7.3 (Oct 11, 2020)
- b2: added exponential back-off around PutBlob() and DeleteBlob() (#679)
- cli: improved restore progress information (#678)
- Cleanup old RPM and DEB packages from 'unstable' repositories (#677)
- Fixed symlink restore behavior on macOS (#673)
- Logging flags (#674)
- restore: support for parallelization (#668)
- cli: 'snapshot create' switched all stderr output to use logger (#663)
- cli: removed confusing default on 'snapshot list' that only lists last 100 snapshots per source (#666)
- testing: use shorter RSA keys to speed up server tests (#665)
- cli: added --force-color and --disable-color flags (#664)
New in Kopia 0.7.2 (Oct 4, 2020)
- Functional changes:
- Fixed retention tag application to incomplete snapshots, automatically apply at checkpoints. (#660)
- Repo: refresh indexes in the background every 15 minutes (#650)
- Upgrade webdav client dependency to include bugfix for the race condition (#651)
- B2: fixed handling of 'no_such_file' to indicate NOT_FOUND (#646)
- Cli: When listing directory that had errors, print error summary at the end. (#643)
- Fixed empty object IDs in checkpoints (#649)
- Restore: improved user experience (#644)
- Fixed few minor data races (#659)
- Eliminated busy loop after snapshot failure (#658)
- Infrastructure Changes:
- Fshasher: truncate timestamps to full seconds when comparing to accomodate filesystems that lose precision (#661)
- Testing: more sleeps in TestFileStorage
- Speed up integration tests (#653)
- Testing: fixed deadlock in faketime_test (#655)
- Makefile: plug in gotestsum for better test output (#652)
- Testing: increase delay in filesystem_storage_test
- Testing: increase test timeout because 90s is often flaky
New in Kopia 0.7.1 (Sep 24, 2020)
- app: changed releaseType to release to fix publishing artifacts to GitHub, 2nd try
- app: changed releaseType to release to fix publishing artifacts to GitHub
- goreleaser: switch draft mode to off as it interferers with scoop publishing
- release: fixed RPM signing by moving all of this to a single invocation of sign.sh
- Remove maintenance lock file on disconnect (#616)
- cli: fixed snapshot delete to support deleting file (not directory) snapshots by object ID (#613)
- Fixed checkpointing to not restart the entire upload process (#594)
- cli: ensure advanced commands are not accidentally used (#611)
- site: edited installation page and switched Download link to point at it (#603)
- Test for directory reuse after GC (#601)
- object: implemented fast concatenation of objects by merging their index entries (#607)
- Splitter throughput improvements (#606)
- object: loadSeekTable helper (#608)
- travis: disable publishing RPM on pull requests since it needs credentials
- build: publishing of scoop Windows packages via goreleaser
- tools: misc fixes to RPM publishing
- tools: clean up repomd directory before updating
- travis: update RPM and APT repos on linux/amd64 even for non-tagged commits
- Trivial test nits (#602)
- testing: Refactor faketime (#597)
- Makefile: fixed publishing logic
- Added RPM repository (#600)
- Go 1.15 (#598)
- object: fixed splitter reset on reuse (#596)
- Update BUILD.md
- Kopia UI improvements for repository management (#592)
- Added support for setting and changing repository client options (description, read-only, hostname, username) (#589)
- Improvements to UX for mounting directories (both CLI and KopiaUI) (#573)
- cli: plumbed through missing --server-cert-fingerprint option (#580)
- upload: fixed ForceHashPercentage behavior to be non-deterministic (#579)
- content: fixed index compaction that would resurrect content entry during full maintenance (#563)
- implemented Cache Directory Tagging Specification + CLI + UI (#565)
- blob: added SetTime() method which may be optionally implemented by blob.Storage (#575)
- apt-publish: publish dummy i386 release, even though we don't make packages for it as Ubuntu on amd64 is sometimes confused and tries downloading i386 anyway (#574)
- Set BuildVersion during repo initialization. (#572)
- cli: ignore trailing / in repository server URL (#569)
- ui: fixed typo (#570)
- site: fixed edit link (#568)
- Doc: fix create B2 repository cmd (#566)
- website: added link to kopia forum (#561)
- Added endurance test which tests kopia over long time scale (#558)
- Tools to help investigate repository structures safely (#553)
- logging: changed default file log level to debug
- Address additional suggestions from fio workload PR #529 (#550)
- Minor doc edits (#549)
- Experimental rclone backend (#545)
- kopia-ui: added ability to connect to kopia server and few other minor tweaks (#546)
- server: pre-read request body to fix HTTP/2 deadlock (#539)
- infra: set up github action to run quick lint on pull requests (#535)
- [Robustness] Add additional fio workloads and fix fio runner (#529)
- apt: update caching parameters after push
- site: updated installation instructions now that we have APT repository (#534)
- [Robustness] Fix for kopia runner and custom working directory (#533)
- kopia-ui: added license ID and description
- apt: fixed matching unstable version numbers
- makefile: fixed BOTO_PATH
- tools: added apt-publish tool that push to APT repository
- goreleaser: tweaks to support APT repositories
- [Robustness] Fswalker should ignore host name (#531)
- [refactor] Move robustness and engine packages (#528)
- Linter upgrade to v1.30.0 (#526)
- testing: increase sleep time to fix flaky arm64 test
- [Robustness] Add command line parameters for kopia snapshotter (#576)
- testing: added performance benchmark (#618)
- Repro and fix for zero-sized snapshot bug (#641)
- Add support for Blake3 hashing (#640)
- Update dependencies (#639)
- Prefer t.TempDir() over makeScratchDir(t) (#612)
- Trivial: update git attributes for Makefile (#638)
- Update build-armhf.yml
- ignorefs: fixed arm-specific linter warning (#637)
- Create build-armhf.yml
- Minor edits for SFTP documentation (#635)
New in Kopia 0.7.0 (Sep 21, 2020)
- SFTP connectivity and docs improvements (#623)
- upload: fixed armhf alignment issue (#632)
- webdav: prevent webdav client race condition (#626)
- cli: don't ask for password if repository is not connected (#627)
New in Kopia 0.7.0 RC 2 (Sep 18, 2020)
- restore: support for symlinks (experimental) (#621)
- testing: increase sleep time to fix flaky arm64 test
- [Robustness] Add command line parameters for kopia snapshotter (#576)
- upload: scan the directory to be uploaded in parallel to estimate the amount of data to be uploaded (#622)
- fuse: changed file read implementation to avoid OOM (#620)
- logging: revamped logs from content manager to be machine parseable (#617)
- testing: added performance benchmark (#618)
New in Kopia 0.7.0 RC 1 (Sep 14, 2020)
- App: changed releaseType to release to fix publishing artifacts to GitHub, 2nd try
- App: changed releaseType to release to fix publishing artifacts to GitHub
- Goreleaser: switch draft mode to off as it interferers with scoop publishing
- Release: fixed RPM signing by moving all of this to a single invocation of sign.sh
- Remove maintenance lock file on disconnect (#616)
- Cli: fixed snapshot delete to support deleting file (not directory) snapshots by object ID (#613)
- Fixed checkpointing to not restart the entire upload process (#594)
- Cli: ensure advanced commands are not accidentally used (#611)
- Site: edited installation page and switched Download link to point at it (#603)
- Test for directory reuse after GC (#601)
- Object: implemented fast concatenation of objects by merging their index entries (#607)
- Splitter throughput improvements (#606)
- Object: loadSeekTable helper (#608)
- Travis: disable publishing RPM on pull requests since it needs credentials
- Build: publishing of scoop Windows packages via goreleaser
- Tools: misc fixes to RPM publishing
- Tools: clean up repomd directory before updating
- Travis: update RPM and APT repos on linux/amd64 even for non-tagged commits
- Trivial test nits (#602)
- Testing: Refactor faketime (#597)
- Makefile: fixed publishing logic
- Added RPM repository (#600)
- Go 1.15 (#598)
- Object: fixed splitter reset on reuse (#596)
- Update BUILD.md
- Kopia UI improvements for repository management (#592)
- Added support for setting and changing repository client options (description, read-only, hostname, username) (#589)
- Improvements to UX for mounting directories (both CLI and KopiaUI) (#573)
- Cli: plumbed through missing --server-cert-fingerprint option (#580)
- Upload: fixed ForceHashPercentage behavior to be non-deterministic (#579)
- Content: fixed index compaction that would resurrect content entry during full maintenance (#563)
- Implemented Cache Directory Tagging Specification + CLI + UI (#565)
- Blob: added SetTime() method which may be optionally implemented by blob.Storage (#575)
- Apt-publish: publish dummy i386 release, even though we don't make packages for it as Ubuntu on amd64 is sometimes confused and tries downloading i386 anyway (#574)
- Set BuildVersion during repo initialization. (#572)
- Cli: ignore trailing / in repository server URL (#569)
- Ui: fixed typo (#570)
- Site: fixed edit link (#568)
- Doc: fix create B2 repository cmd (#566)
- Website: added link to kopia forum (#561)
- Added endurance test which tests kopia over long time scale (#558)
- Tools to help investigate repository structures safely (#553)
- Logging: changed default file log level to debug
- Address additional suggestions from fio workload PR #529 (#550)
- Minor doc edits (#549)
- Experimental rclone backend (#545)
- Kopia-ui: added ability to connect to kopia server and few other minor tweaks (#546)
- Server: pre-read request body to fix HTTP/2 deadlock (#539)
- Infra: set up github action to run quick lint on pull requests (#535)
- [Robustness] Add additional fio workloads and fix fio runner (#529)
- Apt: update caching parameters after push
- Site: updated installation instructions now that we have APT repository (#534)
- [Robustness] Fix for kopia runner and custom working directory (#533)
- Kopia-ui: added license ID and description
- Apt: fixed matching unstable version numbers
- Makefile: fixed BOTO_PATH
- Tools: added apt-publish tool that push to APT repository
- Goreleaser: tweaks to support APT repositories
- [Robustness] Fswalker should ignore host name (#531)
- [refactor] Move robustness and engine packages (#528)
- Linter upgrade to v1.30.0 (#526)
New in Kopia 0.6.4 (Sep 2, 2020)
- content: fixed index compaction that would resurrect content entry during full maintenance
New in Kopia 0.6.3 (Aug 22, 2020)
- Disable blob deletion in 0.6 unless KOPIA_ENABLE_BLOB_DELETION is set to true (#552)
New in Kopia 0.6.2 (Aug 16, 2020)
- Server: pre-read request body to fix HTTP/2 deadlock (backported from master 2b029a4)
New in Kopia 0.6.1 (Aug 13, 2020)
- kopia-ui: roll back electron to v7.1.14 (upgraded in cfa30f2) since it's crashing on Ubuntu (#530)
New in Kopia 0.6.0 (Aug 11, 2020)
- Core:
- Big performance improvements when snapshotting large directories (#331)
- Improvements for dealing with eventually-consistent stores (S3) (#437)
- GC safety improvements to resolve race condition when content is re-referenced when about to be deleted (#420)
- Added OpenCensus (#339)
- Introduced explicit maintenance operations that perform periodic repository cleanup/compaction (#411)
- Disabled automatic compaction on repository opening - moved to maintenance tasks
- Added AsyncWrites to ObjectWriter, which improves performance… (#369)
- Object: ensure that all I objects have a content prefix which improves locality by putting them in q packs
- Deduplicate multiple policies for the same source in policy manager, fixes #391
- Fixed race condition during Open() where we may read incomplete file
- Deprecated NONE algorithm, will not be available for new repositories (#395)
- Server: automatically flush the repository after setting or deleting a policy (#489)
- Snapshot checkpointing (#410)
- Moved creating cache directory from connect to first use (#450)
- Persist relative path to cache if possible, this allows config directory to be partially portable
- Server: implemented 'flush' and 'refresh' API
- Experimental support for remote repository (#427)
- Repo: refactored public API (#318)
- Upload: auto-ignore kopia cache directories when creating snapshots (#524)
- KopiaUI:
- support for multiple repositories + portability (#398)
- highlight snapshot errors (#376)
- Providers:
- Support for gather writes (#373)
- B2: added provider for backblaze b2
- Sftp: add missing options for configuring sftp known_hosts
- S3: add CLI option for disabling tls verification while connecting to s3
- Filesystem: added retry which addresses the macOS race condition
- CLI Improvements:
- Added flags to control progress output
- 'kopia server' made --ui default (#452)
- Allow override of snapshot start time and end time
- Improved 'snapshot delete' usage (#436)
- Remove legacy flags from snapshot create command (#441)
- Support for zip, tar and tar.gz restore outputs (#482)
- Support for repository sync (#522)
- Infrastructure:
- MacOS and Windows KopiaUI builds are now signed
- Robustness testing framework
- Testing: added blob.Storage wrapper that simulates eventual consistency (#434)
- Switched back to using v-prefixed tag names.
- Tests: added smoke test that exercises all combinations of encryption and hashing
- Changelog:
- 9844809 Updated documentation for v0.6.0 release (#525)
- Bb2434d upload: auto-ignore kopia cache directories when creating snapshots (#524)
- 505ab92 Support for repository sync (#522)
- D0e631a fix typo in url (#523)
- B995a1d app: Disabled progress percentage display above 100% (#521)
- Cfa30f2 switched back to using v-prefixed tag names.
- Ff244b5 go.mod: upgrade github.com/klauspost/compress to v1.10.10
- 8ead49b restore: support for zip, tar and tar.gz restore outputs (#482)
- 52e7631 kopia-ui: fixed missing 'log' when clicking 'Launch At Startup'
- 7e9ce61 server: automatically flush the repository after setting or deleting a policy (#489)
- A544e0c kopia-ui: Added logging when waiting for notarization response from Apple.
- 2c21582 kopia-ui: added option to disable TLS and force HTTP
- 73ed5ad kopia-ui: cleaned up console logging
- 8c941cc Fix BUILD.md (#481)
- 0b2d12d kopia-ui: build deb and rpm packages
- Ce5e6dc [Robustness] Add first robustness tests
- 82a2fa0 [Robustness] Add test engine to manage snapshot verification testing (#468)
- D44c01f content: do not short circuit compaction if there's one index but there are contents to delete
- 64a6cb4 parallelwork: fixed error handling, which caused parallel work to never finish on any error
- 1c25341 cli: fixed panic() during snapshot verification caused by refactoring to new Repository interface
- 3843ed6 cli: added content verify --include-deleted flag
- Ed9511b b2: fix GetMetadata() implementation
- 25934a5 content: fixed deletion of cleanup blobs (#462)
- 7975767 server: implemented 'flush' and 'refresh' API
- 293cb10 cosmetic changes to maintenance behavior (#461)
- C13b5f8 Remove extra whitespaces and fix minor typos (#460)
- 85aa4ae Add 'repositories' page (#454)
- Deb1099 Trivial: remove unused logging.Option type
- 6bc47cb content: emit warning whenever the number of index blobs gets too high, which is an indication that maintenance has not run in a long time
- 960c334 maintenance: disabled automatic compaction on repository opening
- 6cb970f Revert "Fix #360: Leverage Go 1.14 testing.T.Cleanup() (#455)" (#458)
- 73b913c Fix #360: Leverage Go 1.14 testing.T.Cleanup() (#455)
- B30da51 Wire up snapshot-store-compare behavior
- D68273a Improvements for dealing with eventually-consistent stores (S3) (#437)
- 9123e3e cli: 'kopia server' made --ui default (#452)
- 130712a Remove unused content.Manager.closed channel (#451)
- E445dc5 repo: moved creating cache directory from connect to first use (#450)
- Fff2b44 Upgrade AWS SDK to 1.31.3 from 1.31.1
- 0642b26 Upgrade github.com/klauspost/compress to 1.10.6
- 0afd1a5 Upgrade github.com/minio/minio
- 97e04d5 Upgrade golang.org/x dependencies
- 0875939 dep: upgrade protobuf dependents (#442)
- 23c1212 Build test/tools in Darwin as well (#447)
- 0c2ad99 Upgrade github.com/minio/minio-go/v6
- 53b82c9 dep: upgrade various dependencies (#445)
- 0a6a0d4 Upgrade AWS SDK to 1.31.1
- Ea681e4 deps: Upgrade compress package to 1.10.5 (#443)
- E493177 Remove legacy flags from snapshot create command (#441)
- 8c4fb53 blob: support for GetMetadata() to get server-side timestamp and blob length (#440)
- Ca28469 cli: improved 'snapshot delete' usage (#436)
- 7e5fc52 removal of unnecessary extra )}
- F3359d7 add sftp and webdav as repositories to "Getting started" documentation page, "Setting Up Repository" chapter
- 993c3bf linter: fixed how linter is installed on Darwin and Linux since it was getting wrong version
- D657415 testing: added blob.Storage wrapper that simulates eventual consistency (#434)
- 200ef57 app: portability fixes (#431)
- Be4b897 Support for remote repository (#427)
- 1377d05 Maintenance changes (#423)
- 4f6dd94 Maintenance: report and record record maintenance run timings (#422)
- A462798 content: added blob-level metadata cache (#421)
- D707f36 GC content repair (#420)
- 2b8e882 filesystem: added retry which addresses the macOS race condition
- 4b4628a Repository maintenance support (#411)
- 573d104 object: ensure that all I objects have a content prefix
- B671bda cli: add missing options for configuring sftp known_hosts
- 14fb49a content: asynchronous cache sweep
- E50b5ec Trivial: Add package alias to silence IDE/gopls warning
- 1f1682b Snapshot checkpointing (#410)
- 70d4c87 cli: improvements to content selection for list/rewrite/stats/verify (#409)
- 057c278 Kopia UI: support for multiple repositories + portability (#398)
- 45195fa appveyor: moved WINDOWS_CERT_SHA1 to secure variable so that we don't sign PRs
- C89b1a7 appveyor: added GH_TOKEN variable, disabled RDP
- 01f52f8 appveyor: fixed secure variable
- 24f6720 enable RDP on appveyor worker until it's stable
- 0017f59 Makefile: fix travis-setup on Windows
- 2055fe3 Makefile: support Windows Appveyor environment (#407)
- 16275a1 Makefile: changed how build number is calculated
- 37b69d8 site: tweaked layout and options for cookie consent popup
- E098464 improved logging for sign.js
- 6ad9273 travis: setup tools to support signing Windows binaries
- 05ea35f testing: retry around B2 test
- 567bd33 added b2 to repository creation ui
- 4bed451 implemented b2 backend
- 3924fb2 Minor cleanup for snapshot time override (#392)
- 6cb9b8f repo: refactored public API (#318)
- 10bb492 repo: deprecated NONE algorithm, will not be available for new repositories (#395)
- 6097781 Support for gather writes (#373)
- B08d394 policy: deduplicate multiple policies for the same source in policy manager, fixes #391
- 393d273 Policy set: fix nil pointer dereference #385 (#387)
- Ea510f3 repo: fixed race condition during Open() where we may read incomplete file
- F2e6d51 repo: persist relative path to cache if possible, this allows config directory to be partially portable
- 9b68a63 Highlight snapshot errors in the UI and CLI (#376)
- 8687f1c object: added AsyncWrites to ObjectWriter, which improves performance… (#369)
- 2e7fe7c travis: switch osx to xcode11.3
- E86e897 fix makefile for windows (#377)
- 6789f8e cli: allow override of snapshot start time and end time
- A1f7a06 Add CLI option for disabling tls verification while connecting to s3 (#370)
- 1df5c81 correct a typo in Doc
- 73e21e0 correct a typo in kopia UI
- 239d809 performance: introduced buf.Pool which helps reuse memory buffers (#345)
- Becf08a testing: added missing condition variable broadcast after finishing flushing, fixes timeouts on Travis
- 4a444e6 content: reduce allocations in index scan
- 169f25c s3: pass blob length to minio in PutBlob()
- B5d4421 upload: added missing writer.Close()
- B102094 splitter: exposed MaxSegmentSize() for splitters
- 4564ed5 encryption: added MaxOverhead to encryptor
- C9877bf performance: refactored content manager to avoid copying data
- 53a5926 content: wait for cache sweep goroutine to close (#361)
- D930f06 cli: added flags to control progress output
- 06fbaba testing: ensure cleanup of content managers, fixes #356
- 272b657 upgrade all dependencies to latest versions
- A8ef0b2 kopia-ui: fixed auto-update
- 4088eb5 site: updated CDN links for cookie consent
- C150767 site: added cookie consent form
- 674543e site: added simple privacy policy
- 331ff07 cli: added number of files that are in the process of being hashed
- 78a921a Rename Simple mutex
- 60d1cc8 [robustness testing] Implement metadata store for test metadata using kopia
- 6b04b42 tests: added smoke test that exercises all combinations of encryption and hashing
- E2a169d hotfix: kopia-0.5.0 did not properly support 256-bit hashes
- 8d452a8 performance: improvements to object manager (#336)
- 526445a CLI: pre-allocated buffers for crypto benchmark (#343)
- E80f553 performance: plumbed through output buffer to encryption and hashing,… (#333)
- 0d1a627 Improved upload performance and memory usage (#331)
- 89c0c6b Refactor CLI stats (#341)
- 71e5563 Fix download link (#342)
- 6ce8410 Added OpenCensus (#339)
- Edc87fc Refactor content stats (#340)
- 9c3d419 Fixed social site logo - alternative #2 (#335)
- 514df69 performance: added wrapper around io.Copy()
- Ec56ed3 Makefile: fixed layering test
- F07d164 Makefile: added layering-test, fixes #324
- 0585232 Add snapshotter interface and kopia implementation
- D9ce3d0 Inject time in Kopia components (#314)
- 5f96b02 testing: added retry helper
- Bd3cace site: changed Community page to make it more obvious how to get to Slack and GitHub
- 88ce341 Trivial cleanup for internal diff (#316)
- 3ebef0a snapshot: removed content stats from snapshot.Stats, which are confusing and incorrect if snapshots are done in parallel
- D862bc2 site: delete news section since it's empty
- 7638d4a site: clarified docs based on Reddit questions, removed stale link to user mailing list which is not in use
New in Kopia 0.6.0 RC 4 (Aug 7, 2020)
- Fix for policy manager not setting labels in some cases (#517)
- Fix #516 - Chown not suported on Windows (#518)
- [Fix #519] Fix pack blob parse in TestRestoreFail (#520)
- cli: auto-upgrade the repository to 0.6.0 (#512)
- Upgrade build-time dependencies (#510)