Commit graph

124 commits

Author SHA1 Message Date
mmetc
dd71f0a866
CI: bump lint version and update configuration (#2901)
* bump golangci-lint to 1.56

* lint (testifylint)

* update lint configuration

* windows test: remove stale code
2024-03-19 10:48:49 +01:00
blotus
6de62a1468
warn if user is using inotify to tail a symlink (#2881) 2024-03-19 10:22:43 +01:00
mmetc
2f49088163
file acquisition: don't bubble error when tailed file disappears (#2903)
* file acquisition: don't bubble error when tailed file disappears
* don't call t.Kill()
* lint (whitespace)
2024-03-18 11:25:45 +01:00
Thibault "bui" Koechlin
b1c09f7512
acquisition : take prometheus level into account (#2885)
* properly take into account the aggregation level of prometheus metrics in acquisition
2024-03-13 14:57:19 +01:00
blotus
44ec3b9e01
file acquis: add mutex to protect access to the internal tail map (#2878) 2024-03-08 13:56:59 +01:00
blotus
5731491b4e
Auto detect if reading logs or storing sqlite db on a network share (#2241) 2024-03-07 14:04:50 +01:00
mmetc
a23fe06d68
remove dependencies on enescakir/emoji, gotest.tools (#2837)
* wrap emoji package in pkg/emoji
* remove dependency on enescakir/emoji
* remove dependency on gotest.tools
* lint (whitespace)
2024-02-23 16:05:01 +01:00
Laurence Jones
f3ea88f64c
Appsec unix socket (#2737)
* Appsec socket

* Patch detection of nil listenaddr

* Allow TLS unix socket

* Merge diff issue
2024-02-21 13:40:38 +00:00
blotus
332af5dd8d
appsec: split return code for bouncer and user (#2821) 2024-02-09 14:39:34 +01:00
Laurence Jones
fa56d35a48
[Loki] Set headers/basic auth if set for queryRange (#2815) 2024-02-09 14:37:49 +01:00
mmetc
f75cdeb239
lint: enalble linter "wastedassign" (#2772) 2024-01-24 17:31:11 +01:00
AlteredCoder
70e8377c0d
Fix appsec evt send order (#2749) 2024-01-17 11:59:31 +01:00
Thibault "bui" Koechlin
685cda545b
fix the reload process for appsec (#2750) 2024-01-17 11:54:44 +01:00
mmetc
08794c5b6d
[appsec] waf tester (#2746) 2024-01-16 11:39:23 +01:00
blotus
6acbcb0a33
Various appsec fixes (#2742) 2024-01-15 16:38:11 +01:00
blotus
e452dc80bd
ignore native modsec rules that were either pass or allow (#2684) 2024-01-15 15:12:02 +01:00
blotus
fd309134a2
log death reason of file reader if available (#2721) 2024-01-15 15:00:49 +01:00
mmetc
5622ac8338
CI: enable testifylint (#2696)
- reverse actual and expected values
 - use assert.False, assert.True
 - use assert.Len, assert.Emtpy
 - use require.Error, require.NoError
 - use assert.InDelta
2024-01-05 15:26:13 +01:00
Thibault "bui" Koechlin
1c03fbe99e
minor waf fixes (#2693) 2024-01-03 17:19:48 +01:00
blotus
b6f272d09a
always set the transaction in the current request (#2682) 2023-12-22 11:44:06 +01:00
blotus
a62e28fdfb
always set inband transaction even if we have no rules (#2681) 2023-12-22 10:18:35 +01:00
blotus
33e3fdabe4
Appsec additional fixes (#2676) 2023-12-21 11:51:04 +01:00
Thibault "bui" Koechlin
51f70e47e3
Minor improvements to hubtest and appsec component (#2656) 2023-12-13 17:45:56 +01:00
AlteredCoder
b1f85693c2
Appsec improvement and fixes after merge (#2645) 2023-12-08 10:25:00 +01:00
Thibault "bui" Koechlin
8cca4346a5
Application Security Engine Support (#2273)
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)

The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)

---------

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc
90d3a21853
CI: use go 1.21.5 (#2640)
* use go 1.21.5
* Simpler go:build directives
2023-12-06 12:38:36 +01:00
blotus
ec53c672dc
Kafka acquisition: warn if no consumer group id and allow to read from a specific partition (#2612) 2023-11-23 10:02:53 +01:00
lperdereau
92f923cfa8
Loki integration #2 (#2306)
* Add support for loki datasource

---------

Co-authored-by: Mathieu Lecarme <mathieu@garambrogne.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-11-22 13:31:39 +01:00
he2ss
947b247a40
kafkaAcquisition: add more debug (#2609)
* kafkaAcquisition: add more debug
2023-11-22 09:35:58 +01:00
blotus
d7ef51e6ba
properly update the cs_syslogsource_parsed_total metric (#2608) 2023-11-22 09:04:23 +01:00
mmetc
5cd4406f5e
typos/grammar (#2561) 2023-11-07 15:07:36 +01:00
blotus
43ef32aa8d
Kafka acquisition: do not create empty events when a read error occurs (#2466) 2023-09-13 13:20:36 +02:00
mmetc
d45bec4047
minor log message improvements (#2455) 2023-09-12 11:04:56 +02:00
mmetc
c588be0842
golangci-lint: use v1.54, remove unnecessary byte/string conversions (#2438) 2023-08-25 16:22:10 +02:00
mmetc
cd9d8f309d
CI: increase test sleep to fix flaky acquisition/file test under win (#2410)
* CI: increase test sleep to attempt fix for flaky windows acquitition/file test

* wip
2023-08-08 16:11:32 +02:00
mmetc
ffadd42779
update dependency on go-cs-lib; drop the pkg/ part (#2393) 2023-07-28 16:35:08 +02:00
blotus
f9ca14f010
add object key in src for S3 acquis (#2342) 2023-07-07 10:09:18 +02:00
blotus
1295de928a
Properly match new files on windows when doing file acquisition (#2329) 2023-07-06 14:45:38 +02:00
mmetc
bd41f855cf
errors.Wrap -> fmt.Errorf (#2317) 2023-06-29 11:34:59 +02:00
mmetc
a910b7beca
non-fatal error if some datasource can't be run (i.e. journalctl but systemd is missing) (#2309)
This on the other hand, gives a new fatal error when there are no valid datasources.
In the previous version, crowdsec kept running with just a warning if no
acquisition yaml or dir were specified.
2023-06-27 10:13:13 +02:00
mmetc
9beb5388cb
errors.Wrap -> fmt.Errorf; clean up imports (#2301) 2023-06-23 14:04:58 +02:00
mmetc
8bfeb7d90d
Update go dependencies (#2293)
- update fatih/color (fix windows issue)
- update mongo-driver (fix build issue)
- go.mod: merge two "require" blocks
- update semver dependency (same version as indirect dep), fix test checks in cscli setup
- remove gotest.tools dependency (use testify, cstest)
- update x/ exp, mod, sys dependencies
2023-06-22 11:31:41 +02:00
mmetc
b2d3520519
decouple bouncer dependencies: use go-cs-lib in test code (#2229) 2023-05-25 15:37:44 +02:00
mmetc
025f14f879
merge system cert pool with own certs (#2226) 2023-05-25 10:10:58 +02:00
mmetc
534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* (#2216)
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
blotus
2701454f23
defaults to inotify to detect changes in file datasource to avoid too many call to stat() (#2181) 2023-05-09 10:03:55 +02:00
blotus
0279e549bd
check if the acquis tomb is dying while processing logs in replay mode for file/s3/docker (#2152) 2023-04-04 13:57:06 +02:00
mmetc
3132aa54b7
Properly load k8s audit configuration (#2158) 2023-04-03 21:55:31 +02:00
mmetc
3fa555fb25
Rename k8s_audit to k8s-audit (easier to type, consistent with labels) (#2153) 2023-04-03 09:53:38 +02:00
blotus
61bea26486
Add transform configuration option for acquisition (#2144) 2023-03-29 16:04:17 +02:00