Commit graph

77 commits

Author SHA1 Message Date
Thibault "bui" Koechlin
717fc97ca0
add SetMeta and SetParsed helpers (#2845)
* add SetMeta and SetParsed helpers
2024-02-14 13:38:40 +01:00
mmetc
f306d59016
logging: full timestamp with timezone in crowdsec.log (#2707)
RFC3339 = "2006-01-02T15:04:05Z07:00" (same as /var/log/syslog)
2024-01-08 21:20:25 +01:00
mmetc
5622ac8338
CI: enable testifylint (#2696)
- reverse actual and expected values
 - use assert.False, assert.True
 - use assert.Len, assert.Emtpy
 - use require.Error, require.NoError
 - use assert.InDelta
2024-01-05 15:26:13 +01:00
AlteredCoder
b1f85693c2
Appsec improvement and fixes after merge (#2645) 2023-12-08 10:25:00 +01:00
Thibault "bui" Koechlin
8cca4346a5
Application Security Engine Support (#2273)
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)

The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)

---------

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc
ec199162dc
iso8601: use yyyy-mm-dd in log timestamps instead of dd-mm-yyyy (#2564)
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-11-24 14:59:28 +01:00
Thibault "bui" Koechlin
1dcf9d1ae1
Improved expr debugger (#2495)
* new expr debugger

---------

Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
2023-11-24 11:10:54 +01:00
Laurence Jones
19de3a8a77
Runtime whitelist parsing improvement (#2422)
* Improve whitelist parsing

* Split whitelist check into a function tied to whitelist, also since we check node debug we can make a pointer to node containing whitelist

* No point passing clog as an argument since it is just a pointer to node we already know about

* We should break instead of returning false, false as it may have been whitelisted by ips/cidrs

* reimplement early return if expr errors

* Fix lint and dont need to parse ip back to string just loop over sources

* Log error with node logger as it provides context

* Move getsource to a function cleanup some code

* Change func name

* Split out compile to a function so we can use in tests. Add a bunch of tests

* spell correction

* Use node logger so it has context

* alternative solution

* quick fixes

* Use containswls

* Change whitelist test to use parseipsource and only events

* Make it simpler

* Postoverflow tests, some basic ones to make sure it works

* Use official pkg

* Add @mmetc reco

* Add @mmetc reco

* Change if if to a switch to only evaluate once

* simplify assertions

---------

Co-authored-by: bui <thibault@crowdsec.net>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-10-16 10:08:57 +01:00
Thibault "bui" Koechlin
e4dcdd2572
fix include_capi filter (#2478) 2023-09-20 11:56:00 +02:00
mmetc
e36df40ba7
pkg/types cleanup (#2398)
* move function GetLineCountForFile from pkg/types to cscli
* move ParseDuration from pkg/types to pkg/database
* remove unused types.Profile, types.RemediationProfile
2023-08-24 09:44:46 +02:00
mmetc
bd41f855cf
errors.Wrap -> fmt.Errorf (#2317) 2023-06-29 11:34:59 +02:00
mmetc
76429f033a
trim pkg/types: move DataSet/GetData to pkg/cwhub, removed unused Clone function (#2271) 2023-06-08 16:49:51 +02:00
mmetc
cf747d65e0
fix missing import (#2275) 2023-06-08 15:49:37 +02:00
mmetc
25bb23d8b7
minor refactor to pkg/types, cscli machines (#2270)
* cleanup: separate ui and logic
* trim some code from pkg/types
2023-06-08 15:08:51 +02:00
mmetc
6096cb3c9b
Move grok_pattern.go away from pkg/types to trim bouncer dependencies (#2269) 2023-06-08 15:07:30 +02:00
mmetc
5b3200173e
don't pre-create log files (not required anymore) (#2267)
The lumberjack package fixed the issue in natefinch/lumberjack#83 (tested with umask 002) and this code is now redundant since we updated the dependency to v2.2.1.
2023-06-07 12:58:35 +02:00
mmetc
396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap (#2235)
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
2023-06-01 16:31:56 +02:00
mmetc
92a9d6c321
types.InSlice() -> slices.Contains() (#2246) 2023-05-31 12:39:22 +02:00
mmetc
534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* (#2216)
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
blotus
772d5b5c32
Add experimental support for re2 (#2138) 2023-03-28 16:26:47 +02:00
Thibault "bui" Koechlin
618be9ff68
properly update the time structure within event (#2122)
* properly update the time structure within event to ensure it works in time-machine

* move LIVE and TIMEMACHINE to pkg/types : less code needs to import leakybucket package, and we avoid duplicating constants
2023-03-16 16:25:50 +01:00
Thibault "bui" Koechlin
5b0fe4b7f1
support for regexps result cache (#2104)
* support for regexps result cache : gcache + xxhash

Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-03-08 16:07:49 +01:00
blotus
85ab9c68a2
Add cscli papi status and cscli papi sync (#2091) 2023-03-03 13:46:28 +01:00
Thibault "bui" Koechlin
a0b264047c
allow user to specify stash's cache strategy (#2037) 2023-02-06 15:42:55 +01:00
Cristian Nitescu
987f119c4b
v3 capi and blocklists links support (#2019)
* v3 model generation

* v3 model generation

* comms

* fixes after master merge

* missing reader close

* use constants defined for types

---------

Co-authored-by: bui <thibault@crowdsec.net>
2023-02-06 14:06:14 +01:00
AlteredCoder
7e871d2278
rename PAPI base URL (#2033) 2023-02-03 12:10:02 +01:00
Thibault "bui" Koechlin
e927717fa0
Polling API Integration (#1715)
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-31 14:47:44 +01:00
Thibault "bui" Koechlin
6fb962a941
Allow parsers to capture data for future enrichment (#1969)
* Allow parsers to capture data in a cache, that can be later accessed via expr helpers (fake multi-line support)
2023-01-11 15:01:02 +01:00
blotus
fdda940ac0
Add Kubernetes audit acquisition (#1767) 2022-12-06 13:47:29 +01:00
mmetc
895691dad1
enabled linters: gocritic, nilerr (#1853) 2022-11-07 10:36:50 +01:00
mmetc
ddd75eae9a
cscli: new tables, --color yes|no|auto option (#1763) 2022-10-07 11:05:35 +02:00
Manuel Sabban
b2130b1593
Fix 1737 (#1738)
* add GetMeta to *types.Event

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-09-07 10:11:39 +02:00
mmetc
414282a2c9
golangci-lint 1.49 and related fixes (#1736) 2022-09-06 13:55:03 +02:00
mmetc
eea07b7a1a
golangci-lint v1.48 and fixes for "usestdlibvars" (#1711) 2022-08-16 09:46:10 +02:00
Thibault "bui" Koechlin
866c200c31
Generic dateparse approach (#1669)
* Allow any parser to suggest a format string for the date to be parsed.

* allow the enricher functions to get the parser's logger so they can inherit the level
2022-07-28 16:41:41 +02:00
mmetc
628d7be1d8
simplify err.Error() to err when used in printf context (#1603) 2022-06-22 15:53:53 +02:00
he2ss
3d6f015211
Add duration expr to add duration formula (#1556)
* add duration expr to add duration formula
2022-06-22 11:29:52 +02:00
mmetc
c78c833400
CI: colored test output, colored crowdsec and crowdsec-api logs, full final db dump for mysql and sqlite (#1596)
* github-ci: color unit test output and logs
* new config option: force_color_logs (useful in CI)
* bats: show sqlite/mysql dump at the end
* removed "-v" (print package names) from "go build"
* general workflow cleanup
2022-06-17 16:12:49 +02:00
mmetc
10585bfecc
enabled linters and fixes for: misspell, predeclared, unconvert, ineffassign, gosimple, govet (#1595) 2022-06-16 14:41:54 +02:00
Thibault "bui" Koechlin
1c0fe09576
Add support for certificate authentication for agents and bouncers (#1428) 2022-06-08 16:05:52 +02:00
mmetc
131ed1b0a7
error reporting (#1501)
* unified error reporting, removed redundancy, tests
2022-05-19 10:48:08 +02:00
blotus
0449ec1868
Windows Support (#1159) 2022-05-17 12:14:59 +02:00
Thibault "bui" Koechlin
e6a35e8714
Improve plugins grouping (alternative to #1424) (#1437)
* Fix races in test (#1446)

Co-authored-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: AlteredCoder <64792091+AlteredCoder@users.noreply.github.com>
2022-04-19 19:12:23 +02:00
mmetc
4b9a0c4ef7
typos (#1453) 2022-04-19 11:25:27 +02:00
Thibault "bui" Koechlin
d8dc01cd94
Revamp unit tests (#1368)
* Revamp unit tests
* Increase coverage
* Use go-acc to get cross packages coverage

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2022-03-29 14:20:26 +02:00
blotus
fb74b2fda7
Improve LAPI performance when under high load (#1273) 2022-02-17 17:52:04 +01:00
mmetc
ad28a979e9
local control flow cleanup (#1215)
removed redundant/unreachable returns, else branches, type declarations, unused variables
2022-02-01 22:08:06 +01:00
Thibault "bui" Koechlin
cc1ab8c50d
switch to utc time everywhere (#1167)
* switch to utc time everywhere


Co-authored-by: alteredCoder <kevin@crowdsec.net>
2022-01-19 14:56:05 +01:00
Thibault "bui" Koechlin
a88848009a
fix default perms for log file (#1177)
* fix default perms
2022-01-18 16:54:02 +01:00
Thibault "bui" Koechlin
6e92da76ad
lapi to capi : allow push of tainted/custom/manual decisions (#1154)
* add console command to control signal sharing
* modify metrics endpoint to add lastpush

Co-authored-by: alteredCoder <kevin@crowdsec.net>
2022-01-13 16:46:16 +01:00