Thibault "bui" Koechlin
8cca4346a5
Application Security Engine Support ( #2273 )
...
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)
The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)
---------
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc
ffcab0b2bc
Refactor hub management and cscli commands ( #2545 )
2023-11-24 15:57:32 +01:00
Thibault "bui" Koechlin
1dcf9d1ae1
Improved expr debugger ( #2495 )
...
* new expr debugger
---------
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
2023-11-24 11:10:54 +01:00
Laurence Jones
19de3a8a77
Runtime whitelist parsing improvement ( #2422 )
...
* Improve whitelist parsing
* Split whitelist check into a function tied to whitelist, also since we check node debug we can make a pointer to node containing whitelist
* No point passing clog as an argument since it is just a pointer to node we already know about
* We should break instead of returning false, false as it may have been whitelisted by ips/cidrs
* reimplement early return if expr errors
* Fix lint and dont need to parse ip back to string just loop over sources
* Log error with node logger as it provides context
* Move getsource to a function cleanup some code
* Change func name
* Split out compile to a function so we can use in tests. Add a bunch of tests
* spell correction
* Use node logger so it has context
* alternative solution
* quick fixes
* Use containswls
* Change whitelist test to use parseipsource and only events
* Make it simpler
* Postoverflow tests, some basic ones to make sure it works
* Use official pkg
* Add @mmetc reco
* Add @mmetc reco
* Change if if to a switch to only evaluate once
* simplify assertions
---------
Co-authored-by: bui <thibault@crowdsec.net>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-10-16 10:08:57 +01:00
Laurence Jones
ff7acd3347
Reset grokky once all patterns are compiled as we do not need to hold them in memoory ( #2420 )
2023-10-13 12:53:42 +01:00
Laurence Jones
64deeab1ec
Fix PO expr whitelist ( #2471 )
2023-09-19 12:51:03 +01:00
mmetc
d45bec4047
minor log message improvements ( #2455 )
2023-09-12 11:04:56 +02:00
Laurence Jones
86d9384954
Whitelist reason ( #2439 )
...
* Update node.go
Dont update whitelist reason if event is whitelisted
* oops
2023-08-23 14:51:37 +01:00
Laurence Jones
0334a9afe8
Add method name to child logger so we can see which function is erroring when in enrichers ( #2411 )
2023-08-08 13:38:11 +01:00
mmetc
ffadd42779
update dependency on go-cs-lib; drop the pkg/ part ( #2393 )
2023-07-28 16:35:08 +02:00
mmetc
a01ce18b98
replace imports of path with path/filepath ( #2330 )
2023-07-26 10:29:58 +02:00
mmetc
b9a3acb03f
light pkg/parser cleanup ( #2279 )
...
* pkg/parser: clean up imports
* remove duplicate import
* simplify boolean expression
* don't check length before range
* if..else if.. -> switch/case
* errors.Wrap -> fmt.Errorf
* typo, lint
* redundant break
2023-06-13 13:16:13 +02:00
mmetc
6096cb3c9b
Move grok_pattern.go away from pkg/types to trim bouncer dependencies ( #2269 )
2023-06-08 15:07:30 +02:00
mmetc
396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap ( #2235 )
...
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
2023-06-01 16:31:56 +02:00
blotus
6720d89845
fix lock when dumping the parsing state in explain mode ( #2234 )
2023-05-26 15:23:50 +01:00
mmetc
534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* ( #2216 )
...
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
Thibault "bui" Koechlin
77f2968267
fix the behavior of json unmarshal to not return the full map ( #2199 )
2023-05-16 09:10:38 +02:00
Laurence Jones
424215f228
Add ParseKV helper and rework UnmarshalJSON as a proper helper ( #2184 )
2023-05-12 09:43:01 +02:00
blotus
772d5b5c32
Add experimental support for re2 ( #2138 )
2023-03-28 16:26:47 +02:00
blotus
1095f6c875
use expr.Function for custom functions instead of passing them in the env ( #2133 )
2023-03-28 10:49:01 +02:00
Thibault "bui" Koechlin
a3e5f0a3a0
fix dateparse ( #2135 )
2023-03-22 08:20:21 +01:00
Thibault "bui" Koechlin
618be9ff68
properly update the time structure within event ( #2122 )
...
* properly update the time structure within event to ensure it works in time-machine
* move LIVE and TIMEMACHINE to pkg/types : less code needs to import leakybucket package, and we avoid duplicating constants
2023-03-16 16:25:50 +01:00
Thibault "bui" Koechlin
855f9e6f8d
protect map w/ mutex to avoid concurrent map writes with cscli explain when having many concurrent parser routines ( #2113 )
2023-03-16 11:01:25 +01:00
mmetc
e161507d08
Lint (type inference): remove redundant type declarations ( #2111 )
2023-03-09 11:56:02 +01:00
Thibault "bui" Koechlin
5b0fe4b7f1
support for regexps result cache ( #2104 )
...
* support for regexps result cache : gcache + xxhash
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-03-08 16:07:49 +01:00
Thibault "bui" Koechlin
a0b264047c
allow user to specify stash's cache strategy ( #2037 )
2023-02-06 15:42:55 +01:00
Thibault "bui" Koechlin
6fb962a941
Allow parsers to capture data for future enrichment ( #1969 )
...
* Allow parsers to capture data in a cache, that can be later accessed via expr helpers (fake multi-line support)
2023-01-11 15:01:02 +01:00
AlteredCoder
185f9ad541
Alert context ( #1895 )
...
Co-authored-by: bui <thibault@crowdsec.net>
2023-01-04 16:50:02 +01:00
Laurence Jones
fd1c38811e
Add parse unix to dateparse enricher ( #1958 )
...
Add parse unix is we do have a strTime but wasnt parsed using convential golang time
2022-12-30 12:47:14 +00:00
mmetc
cc228f1868
Typos, grammar ( #1905 )
2022-12-06 15:55:27 +01:00
blotus
fdda940ac0
Add Kubernetes audit acquisition ( #1767 )
2022-12-06 13:47:29 +01:00
mmetc
104f5d1fe6
lint: error handling cleanup ( #1877 )
2022-11-29 09:16:07 +01:00
mmetc
895691dad1
enabled linters: gocritic, nilerr ( #1853 )
2022-11-07 10:36:50 +01:00
mmetc
1d9f861f28
unit tests: always capture testcase variable -> allow parallel testing ( #1797 )
2022-10-10 10:48:26 +02:00
Manuel Sabban
1f06f242cc
fix https://github.com/crowdsecurity/crowdsec/issues/1746 ( #1749 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-09-14 15:26:26 +02:00
mmetc
414282a2c9
golangci-lint 1.49 and related fixes ( #1736 )
2022-09-06 13:55:03 +02:00
Thibault "bui" Koechlin
866c200c31
Generic dateparse approach ( #1669 )
...
* Allow any parser to suggest a format string for the date to be parsed.
* allow the enricher functions to get the parser's logger so they can inherit the level
2022-07-28 16:41:41 +02:00
AlteredCoder
39da36361c
Get geoip Country from other objects if not present ( #1659 )
2022-07-12 15:26:34 +02:00
AlteredCoder
5f62d738fc
Add no-capi flag and review some logs ( #1628 )
...
* Add no-capi flag and review some logs
2022-07-01 16:56:13 +02:00
AlteredCoder
a4f4eabf0a
support yml file ( #1605 )
2022-06-22 17:01:27 +02:00
mmetc
628d7be1d8
simplify err.Error() to err when used in printf context ( #1603 )
2022-06-22 15:53:53 +02:00
he2ss
3d6f015211
Add duration expr to add duration formula ( #1556 )
...
* add duration expr to add duration formula
2022-06-22 11:29:52 +02:00
mmetc
d71279f023
added flag crowdsec --warning ( #1461 )
2022-06-22 09:38:23 +02:00
mmetc
10585bfecc
enabled linters and fixes for: misspell, predeclared, unconvert, ineffassign, gosimple, govet ( #1595 )
2022-06-16 14:41:54 +02:00
blotus
4b311684ab
Add more JSON expr helpers ( #1576 )
2022-06-08 12:15:29 +02:00
mmetc
799cc82bb5
functional tests, minor refactoring and lint/cleanup ( #1570 )
...
* cmd/crowdsec: removed log.Fatal()s, added tests and print error for unrecognized argument
* updated golangci-lint to v1.46
* lint/deadcode: fix existing issues
* tests: cscli config backup/restore
* tests: cscli completion powershell/fish
* err check: pflags MarkHidden()
* empty .dockerignore (and explain the reason)
* tests, errors.Wrap
* test for CS_LAPI_SECRET and minor refactoring
* minor style changes
* log cleanup
2022-06-06 15:24:48 +02:00
Thibault "bui" Koechlin
0483b9c641
do not spew.Sdump() the invalid node on error. It leads to huge memory usage, especially if the parsers refers ie. datafile ( #1550 )
2022-05-20 13:29:47 +02:00
AlteredCoder
1e1741aa45
Allow to set static to a pointer and add IsIPV6 helper ( #1540 )
...
* Allow to set static to a pointer and add IsIPV6 helper
2022-05-19 16:28:25 +02:00
Cristian Nitescu
a49b023a28
GetExprEnv usage optimization ( #1515 )
...
* avoid multiples calls to GetExprEnv
* cache ExprEnv in node process
* use global expression env
* remove block profile rate
2022-05-17 10:50:37 +02:00
Greg Myers
0f4ab71f01
Fix typos in docs, comments, code ( #1483 )
2022-04-27 11:04:12 +02:00
