Commit graph

66 commits

Author SHA1 Message Date
mmetc
1509c2d97c
pkg/cwhub refact (#2606)
* Separate Item and ItemState; fill BelongsToCollections with all ancestors and for uninstalled items too
* fix "installed parents" check when removing an item
* keep BelongsToCollections in order (case insensitive)
2023-11-21 11:06:59 +01:00
mmetc
6b317f0723
Refact cwhub: simplify tree scan and dependency checks (#2600)
* method rename: GetInstalledItemsAsString() -> GetInstalledItemNames()
* use path package
* Comments and method names
* Extract method Item.setVersionState() from Hub.itemVisit()
* refact localSync(), itemVisit() etc.
* fix check for cyclic dependencies, with test
2023-11-20 11:41:31 +01:00
mmetc
56ad2bbf98
Refact cwhub: item removal with shared dependencies (#2598)
* Iterate over sub-items in Remove(), not in disable() -- fix shared dependency issue
* Increase hub download timeout to 2 minutes
2023-11-16 17:00:51 +01:00
mmetc
d9b0d440bf
Refact cwhub (#2596)
* unused param
* (slightly) simpler ListItems() -> listItems()
* listItems(): always showHeader, deduce showType
ref. https://github.com/crowdsecurity/crowdsec/issues/1068
* simplify Item.disable()
also, .tainted and .installed do not need a default since they are always in the json output now
* Drop unused parameters
2023-11-16 11:09:49 +01:00
mmetc
79d019f9a2
Refact cwhub / sort cscli output, case insensitive (#2593)
* dead code: unknown localVersion now defaults to "?"
* skip type declaration; whitespace
* sync: next item if invalid cpath
* func tests for install --force and --ignore
* shorter test names
* sort cscli <itemtype> output, with tests
* cscli: refact hub sort code
2023-11-15 16:59:30 +01:00
mmetc
4a6fd338e0
replace 'timeout' helper with async python script; allow hub preload in func tests; improve item removal (#2591)
* replace 'timeout' helper with async python script; allow hub preload in func tests; improve item removal
* func tests: cscli hub update/upgrade
* docker test update
* Update docker entrypoint to disable items with --force

The --force flag was not transmitted to cscli, but is required after the hub refact
to disable items inside installed collections
2023-11-14 17:36:07 +01:00
Marco Mariani
120f7cf578 Merge branch 'master' into hub-1.5.6 2023-11-14 15:04:50 +01:00
mmetc
042d316fab
Refact cwhub: remove global hub, func test improvements (#2588)
* csConfig.Cscli is always loaded now, configuration paths too
* Remove global/singleton hub instance
* read {index_path} from config instead of assuming {hub_dir}/.index.json
* fix segfault with cscli explain when no parser is installed
* cscli: help text
* hub download timeout 20 sec
* reduce log verbosity
* allow func tests with empty hub or pre-download
* cscli <itemtype> remove --all --purge
2023-11-14 14:58:36 +01:00
mmetc
9d7ed12950
Refact cwhub (#2586)
* Inspect item: always show tainted, installed, etc. when false
* cleanup, comments, unused stuff
* download collection content after downloading dependencies, avoid duplicate call
* Return instances from Item.SubItems()
* shorter i/o code
* inline / simplify getData()
* Handle timeout connections when downloading from hub or data
2023-11-10 10:25:29 +01:00
mmetc
ab8de19506
Refact cwhub: move methods from hub to item (#2585)
* Add back pointer Item.hub
* Hub.enableItem() -> Item.enable()
* Rename variable i -> idx (i is used for item instances)
* Move Hub.purgeItem() -> Item.purge()
* Move Hub.disableItem() -> Item.disable()
* Move Hub.downloadItem() -> Item.download()
* Move Hub.downloadLatest() -> Item.downloadLatest()
* Move Hub.DownloadDataIfNeeded() -> Item.DownloadDataIfNeeded()
* Move Hub.InstallItem() -> Item.Install()
* Move Hub.RemoveItem() -> Item.Remove()
* Move Hub.UpgradeItem() -> Item.Upgrade()
* store hub items as pointers
* No need to re-add items to the hub if we use pointers
* Fix parameter calling order + regression test
2023-11-09 15:19:38 +01:00
mmetc
f4b5bcb865
Refact cwhub: version comparison and branch selection (#2581)
* simplify GetItemByPath
* hub: sort version numbers by semver
* replace golang.org/x/mod/semver with github.com/Masterminds/semver/v3 (would not compare correctly)
* fix nil dereference with tainted items
* update tests for collections, postoverflows
* fix nil deref
* don't fallback to master if hub is not found, improve message
* explicit message for unknown version / tainted collections
2023-11-08 13:21:59 +01:00
guangwu
ddd6ee8e42
fix: typo (#2582)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-11-08 09:26:34 +01:00
Manuel Sabban
4934fce769
update gantsign.golang name (#2558) 2023-11-07 14:53:14 +01:00
mmetc
bfd94ceda7
make ParserIndex(), DownloadIndex() private methods (#2579)
* unnecessary pointer type
* ParseIndex() as hub method, don't collect missing items since they are never used
* don't export hub.parseIndex(), hub.downloadIndex()
2023-11-07 10:27:33 +01:00
mmetc
41d19de092
Refact cwhub (#2578)
* Fix suggest functional tests
* comments
* non-empty SubItems() implies collections type
* use "slices" from stdlib
* No need to repeat author field in the index -- take it from the item key
2023-11-06 17:35:33 +01:00
mmetc
590a19b768
Refact pkg/cwhub: constructor, cscli output
* Single constructor: NewHub() to replace InitHub(), InitHubUpdate()
* sort cscli hub list output
* log.Fatal -> fmt.Errorf
2023-10-31 12:47:39 +01:00
mmetc
17662e59a9
Refact pkg/cwhub, cscli: hub upgrades (#2568)
* fix bats test for "upgrade all items"
* refact UpgradeConfig() -> UpgradeItem(): one item only
* refact RemoveMany() -> RemoveItem()
* Computed value: Item.Local -> Item.IsLocal()
* refact url/branch configuration with LocalHubCfg/RemoteHubCfg
2023-10-30 17:23:50 +01:00
mmetc
ac98256602
Refact pkg/cwhub, cmd/crowdsec-cli (#2557)
- pkg/cwhub: change file layout, rename functions
 - method Item.SubItems
 - cmd/crowdsec-cli: generic code for hub items
 - cscli: removing any type of items in a collection now requires --force
 - tests
2023-10-20 14:32:35 +02:00
mmetc
be6555e46c
Refact pkg/csconfig, HubCfg (#2552)
- rename csconfig.Hub -> HubCfg
 - move some Load*() functions to NewConfig()
 - config.yaml: optional common section
 - remove unused working_dir
2023-10-18 09:38:33 +02:00
mmetc
325003bb69 Refact cscli item listing, tests (#2547)
* hub diet; taint tests
* cmd/crowdsec-cli: split utils.go, moved cwhub.GetHubStatusForItemType()
* cscli: refactor hub list commands, fix edge cases
2023-10-17 16:12:41 +02:00
mmetc
f496bd1692 bats: more cscli hub tests (#2541)
- updated logs and user messages
- added func tests for all the items: install, remove, upgrade, list
- rewritten taint tests for collections
- removed redundant csconfig.LoadPrometheus()
2023-10-17 16:12:41 +02:00
mmetc
7db5bf8979 pkg/csconfig: set prometheus address:port defaults (#2533)
We set these default in one place (after loading the configuration)
instead of leaving that to both metric server and consumer.
2023-10-17 16:12:41 +02:00
mmetc
a254b436c7
use go 1.12.3 (#2535) 2023-10-12 16:28:24 +02:00
mmetc
61d4ccbfdd
use go 1.21.1 (#2418)
* use go 1.21.1, require 1.21
* import "slices" from stdlib
* allow codeql to set version number from tags
* codeql: custom WASM build - the automated one can silently fail
2023-10-04 13:01:57 +02:00
mmetc
bfda483c0a
fix issue #2499 - nil dereference while using capi whitelists (#2501) 2023-10-02 11:42:17 +02:00
mmetc
3cb9dbdb21
notification-email: configurable timeouts (#2465)
* configurable timeouts
* parse email timeouts as duration string
* add helo_host to email.yaml
* move html and body tags outside of the loops
* added quotes to href=.., and formatting test
2023-09-29 16:59:06 +02:00
mmetc
95ed308207
cscli setup: accept stdin; fix proftpd detection test and service unmask (#2496) 2023-09-29 12:58:35 +02:00
mmetc
b2212f4225
Use go 1.20.8 (#2473) 2023-09-19 13:21:55 +02:00
mmetc
fd94e2c056
refactor alert/decisions insert/update to avoid database locking in bulk operations (#2446) 2023-09-04 14:21:45 +02:00
mmetc
22146eb3e4
fix "cscli console disable --all"; cleanup "cscli console" command (#2444) 2023-08-29 11:44:23 +02:00
mmetc
b562103024
Make: build with debug symbols in func tests or if DEBUG=1; drop BUILD_VENDOR_FLAGS (#2443) 2023-08-28 15:58:26 +02:00
mmetc
32f196a774
use go 1.20.7 (#2409) 2023-08-25 16:24:04 +02:00
mmetc
f2154e362b
update functional tests for build pipeline (#2442) 2023-08-25 16:15:28 +02:00
mmetc
2aa55e9444
move plugins/notifications/* to cmd/notification-* (#2429)
This ensures keeping all dependencies in sync, and simplifies
packaging under freebsd/gentoo/etc because there is a single
vendor directory.
2023-08-24 09:46:25 +02:00
mmetc
644c767019
cscli decisions list -o json => [] instead of null; same for alerts (#2397) 2023-08-03 12:51:50 +02:00
mmetc
5cb7013575
Check cscli preconditions with crowdsec-cli/require package (#2388) 2023-07-27 17:02:20 +02:00
blotus
77d58652a3
add sentinel notification plugin (#2268) 2023-07-25 15:07:10 +02:00
Manuel Sabban
f12ff3dfed
CI: update ansible requirements (#2364) 2023-07-24 15:35:07 +02:00
mmetc
202112bcae
CI: test with postgres 15 (#2149)
Postgres 15 restricts the default privileges for the public schema. We set crowdsec_test as owner which is shorter than granting permissions explicitly.
2023-07-24 11:56:04 +02:00
mmetc
b6b6fd026b
typo fix, uppercase 'API', adjusted log level (#2361) 2023-07-21 23:23:24 +02:00
mmetc
e73ceafdba
Use go 1.20.6 (#2358) 2023-07-18 09:51:32 +02:00
mmetc
486b56d1ed
CI: reduce test verbosity; set PKG_CONFIG_PATH for re2 in rpm distros (#2331)
* wip

* wip

* go install with commit hash
2023-07-05 17:45:31 +02:00
mmetc
73f71a0aa3
tests: vagrant refactoring (#2328) 2023-07-04 12:26:32 +02:00
mmetc
17cd792826
CI: update ansible tests for re2 (#2318) 2023-06-29 16:35:19 +02:00
mmetc
ebe25d7653
func tests: install dependencies from make, log test helpers (#2314) 2023-06-28 10:07:05 +02:00
mmetc
85839b0199
support for stdin with "cscli decision import" and raw values (#2291)
and remove Origin from the struct, which was ignored anyway
2023-06-27 14:29:42 +02:00
mmetc
a910b7beca
non-fatal error if some datasource can't be run (i.e. journalctl but systemd is missing) (#2309)
This on the other hand, gives a new fatal error when there are no valid datasources.
In the previous version, crowdsec kept running with just a warning if no
acquisition yaml or dir were specified.
2023-06-27 10:13:13 +02:00
mmetc
d4c0643122
CI: add fedora-37, -38 to vagrant tests (#2299) 2023-06-23 13:59:24 +02:00
mmetc
fddf597040
errors.Wrap -> fmt.Errorf; clean up imports (#2297) 2023-06-22 15:01:34 +02:00
Laurence Jones
062f71fb92
CI: vagrant configuration for debian 12 (#2285) 2023-06-19 21:18:29 +02:00