Commit graph

618 commits

Author SHA1 Message Date
mmetc
3253b16f0f
Refact pkg/cwhub (part 2) (#2513)
* remove globals for walker callback
* extract method getItemInfo()
* code dedup, if/else -> switch
* dedent: happy path
* remove target variable
2023-10-04 11:17:35 +02:00
mmetc
d39131d154
Refact pkg/cwhub (part 1) (#2512)
* wrap errors, whitespace
* remove named return
* reverse CheckSuffix logic, rename function
* drop redundant if/else, happy path
* log.Fatal -> fmt.Errorf
* simplify GetItemMap, AddItem
* var -> const
* removed short-lived vars
* de-duplicate function and reverse logic
2023-10-04 10:34:10 +02:00
mmetc
8b5ad6990d
lint: pkg/cwhub (#2510)
no functional changes
 
 - reformat
 - comments
 - whitespace
 - removed a dot or two in log messages
 - some "var x=y" -> x:=y
2023-10-03 11:20:56 +02:00
mmetc
6dadfcb2ef
refact: simplify hubtest CopyDir() (#2509) 2023-10-03 11:17:02 +02:00
mmetc
bfda483c0a
fix issue #2499 - nil dereference while using capi whitelists (#2501) 2023-10-02 11:42:17 +02:00
Laurence Jones
b8e6bd8c9a
[Explain] s02 can cause panic if empty (#2486)
* Add parsers length check as it can panic is enrich is empty

* Lets get smarter and loop backwards to find last successful stage

* Shorten code

---------

Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-09-29 12:03:56 +01:00
mmetc
95ed308207
cscli setup: accept stdin; fix proftpd detection test and service unmask (#2496) 2023-09-29 12:58:35 +02:00
Thibault "bui" Koechlin
8f6659a2ec
fix the float comparison by using Abs(a,b) < 1e-6 approach (IEEE 754). Move the initializiation of expr helpers (#2492) 2023-09-28 17:22:00 +02:00
Laurence Jones
37c0c067a8
cscli hubtest whitelist (#2479)
* Initial tests

* Always print whitelist as we can compare if we mess up the opposite way
2023-09-20 16:42:19 +01:00
Thibault "bui" Koechlin
e4dcdd2572
fix include_capi filter (#2478) 2023-09-20 11:56:00 +02:00
mmetc
ac01faf483
strip '=' signs from encoded api keys (#2472)
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-09-19 14:00:23 +02:00
Thibault "bui" Koechlin
4c08e1e68c
exclude 'lists' too if we exclude CAPI (#2474) 2023-09-19 13:56:22 +02:00
mmetc
d5b6f2974b
Avoid sending nil body with metrics (#2470) 2023-09-19 13:53:50 +02:00
Laurence Jones
64deeab1ec
Fix PO expr whitelist (#2471) 2023-09-19 12:51:03 +01:00
blotus
43ef32aa8d
Kafka acquisition: do not create empty events when a read error occurs (#2466) 2023-09-13 13:20:36 +02:00
Thibault "bui" Koechlin
0040569fa9
if 'include capi' is false, only exclude capi alerts instead of assuming they necessarily have attached decisions (#2435) 2023-09-12 11:19:36 +02:00
mmetc
d45bec4047
minor log message improvements (#2455) 2023-09-12 11:04:56 +02:00
mmetc
fd94e2c056
refactor alert/decisions insert/update to avoid database locking in bulk operations (#2446) 2023-09-04 14:21:45 +02:00
Laurence Jones
aff80a2863
Add html escape function so it can be invoked from template (#2451) 2023-09-04 09:49:39 +01:00
mmetc
25868f27de
option db_client.decision_bulk_size (#2440) 2023-08-25 17:05:17 +02:00
mmetc
c588be0842
golangci-lint: use v1.54, remove unnecessary byte/string conversions (#2438) 2023-08-25 16:22:10 +02:00
mmetc
2aa55e9444
move plugins/notifications/* to cmd/notification-* (#2429)
This ensures keeping all dependencies in sync, and simplifies
packaging under freebsd/gentoo/etc because there is a single
vendor directory.
2023-08-24 09:46:25 +02:00
mmetc
e36df40ba7
pkg/types cleanup (#2398)
* move function GetLineCountForFile from pkg/types to cscli
* move ParseDuration from pkg/types to pkg/database
* remove unused types.Profile, types.RemediationProfile
2023-08-24 09:44:46 +02:00
Laurence Jones
86d9384954
Whitelist reason (#2439)
* Update node.go

Dont update whitelist reason if event is whitelisted

* oops
2023-08-23 14:51:37 +01:00
mmetc
6a6501691a
change behavior of flag disable_http_retry_backoff (#2426)
now it does not attempt any retry, instead of attempting all retries
immediately

example: cannot reach LAPI

Before:

$ CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF=true cscli decisions list
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 4 retries left
INFO[27-07-2023 10:44:44] retrying in 0 seconds (attempt 2 of 5)
[...]
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 1 retries left
INFO[27-07-2023 10:44:44] retrying in 0 seconds (attempt 5 of 5)
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 0 retries left
FATA[27-07-2023 10:44:44] Unable to list decisions : performing request: Get "http://localhost:8080/v1/alerts?has_active_decision=true&include_capi=false&limit=100": could not get jwt token: Post "http://localhost:8080/v1/watchers/login": dial tcp [::1]:8080: connect: connection refused

After:

$ CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF=true ./test/local/bin/cscli decisions list
FATA[11-08-2023 16:49:58] unable to retrieve decisions: performing request: Get "http://127.0.0.1:8080/v1/alerts?has_active_decision=true&include_capi=false&limit=100": could not get jwt token: Post "http://127.0.0.1:8080/v1/watchers/login": dial tcp 127.0.0.1:8080: connect: connection refused
2023-08-16 21:04:07 +02:00
mmetc
afeb541eac
apic: minor refactoring (#2415)
* apic: minor refactoring

* Add whitelist length check

If user configures the file but fails to define and actual whitelist we should check length to save allocs

* Init with length from file

* extract loop method from ApplyApicWhitelists

* pass pointer

* extract loop method updateBlocklist

---------

Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-08-10 13:03:47 +02:00
Laurence Jones
93c22f29cf
Unmarshal Json (#2414)
Log the actual line that caused an error to help debugging
2023-08-09 09:42:08 +01:00
Manuel Sabban
d6361d0a40
conditional overflow doesn't overflow on capacity (#2412)
* conditional overflow doesn't overflow on capacity

* typo
2023-08-08 16:12:50 +01:00
mmetc
cd9d8f309d
CI: increase test sleep to fix flaky acquisition/file test under win (#2410)
* CI: increase test sleep to attempt fix for flaky windows acquitition/file test

* wip
2023-08-08 16:11:32 +02:00
Laurence Jones
0334a9afe8
Add method name to child logger so we can see which function is erroring when in enrichers (#2411) 2023-08-08 13:38:11 +01:00
Laurence Jones
a18df9c3bb
Add bouncers prune command (#2379)
* Add bouncers prune command

* No point overloading functions

* Add prune to list of commands

* change all short desc to be similar, and made it really really clear when pruning it is not recoverable

* Dont use log. and dont return error on user input to abort
2023-07-28 15:37:39 +01:00
mmetc
ffadd42779
update dependency on go-cs-lib; drop the pkg/ part (#2393) 2023-07-28 16:35:08 +02:00
Laurence Jones
55247cd46a
Add machines prune command (#2011)
* Add machines prune command

* Fix scope variable for naming scheme

* Add some freshness and add new features

* Fix force and fix duration if less than 60

* Allow duration to be more readable

* Fix description

* Improve func wording and make int machines length

* No point overloading functions

* Add prune to list of commands

* Check if GID is already the group if so no need to chown

* Revert "Check if GID is already the group if so no need to chown"

This reverts commit c7cef1773e.

* change all short desc to be similar, and made it really really clear when pruning it is not recoverable

* Better examples

* Match bouncer like for like

* Fix merge error

* Dont use log. and dont return error on user input to abort
2023-07-28 15:23:47 +01:00
mmetc
ae53c0f1cc
fix "crowdsec-cli/require" log verbosity (#2390) 2023-07-28 09:56:20 +02:00
Thibault "bui" Koechlin
718721b341
fix a confusing debug message (#2386)
* fix a confusing debug message

* make CTIHelper simply log the error to avoid failing template rendering
2023-07-28 09:52:21 +02:00
mmetc
5cb7013575
Check cscli preconditions with crowdsec-cli/require package (#2388) 2023-07-27 17:02:20 +02:00
mmetc
a01ce18b98
replace imports of path with path/filepath (#2330) 2023-07-26 10:29:58 +02:00
Laurence Jones
389ea4293f
Add metabase version override and update (#2370)
* Add version override and update

* Ooppsie

* Quick fix

* fgs copilot

* Allow user to overwrite image, add warning for exposing metabase and general cleanup

* One ix

* Default image if not found in config, and add a warning to remove and update

* Reorder check system memory checks so it inline with @mmetc best pratices

* No need for err

* Clean up some group code

* Change ipv6 as [] seems to wildcard

* Split loopback warn and disclaimer. Add force yes to start to allow user to accept disclaimer by default

* All cmd commands are RunE clean up

* Update flag name and dont allow a shorthand
2023-07-25 14:21:25 +01:00
mmetc
395cace69f
fix double push of metrics by properly handling tickers (#2374) 2023-07-25 12:19:26 +02:00
blotus
7106d396dc
expose the FormatAlert function to other packages (#2248) 2023-07-25 09:55:39 +02:00
AlteredCoder
b52b4252c1
scenario labels to map string interface (#2201)
* labels are now map string interface

* restore api url

---------

Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-07-24 15:19:28 +02:00
mmetc
46fff0b544
Update dependency: docker/docker (#2360) 2023-07-24 11:53:33 +02:00
mmetc
b6b6fd026b
typo fix, uppercase 'API', adjusted log level (#2361) 2023-07-21 23:23:24 +02:00
Manuel Sabban
9ac5aeda79
fix the ci by adding the ability to enforce event ordering (#2347)
* fix the ci by adding the ability to enforce event ordering
2023-07-20 11:41:30 +02:00
blotus
f9ca14f010
add object key in src for S3 acquis (#2342) 2023-07-07 10:09:18 +02:00
blotus
1295de928a
Properly match new files on windows when doing file acquisition (#2329) 2023-07-06 14:45:38 +02:00
mmetc
c10bca93df update dependencies on go-plugin and go-hclog (#2341)
* update dependencies on go-plugin and go-hclog
* bump logrus (panic fix)
* implement HCLogAdapter.Getleve() to satisfy the new interface
2023-07-06 12:01:07 +02:00
mmetc
9967d60987
errors.Wrap -> fmt.Errorf (#2333) 2023-07-06 10:14:45 +02:00
mmetc
17cd792826
CI: update ansible tests for re2 (#2318) 2023-06-29 16:35:19 +02:00
mmetc
bd41f855cf
errors.Wrap -> fmt.Errorf (#2317) 2023-06-29 11:34:59 +02:00