mmetc
6a6501691a
change behavior of flag disable_http_retry_backoff ( #2426 )
...
now it does not attempt any retry, instead of attempting all retries
immediately
example: cannot reach LAPI
Before:
$ CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF=true cscli decisions list
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 4 retries left
INFO[27-07-2023 10:44:44] retrying in 0 seconds (attempt 2 of 5)
[...]
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 1 retries left
INFO[27-07-2023 10:44:44] retrying in 0 seconds (attempt 5 of 5)
ERRO[27-07-2023 10:44:44] error while performing request: dial tcp [::1]:8080: connect: connection refused; 0 retries left
FATA[27-07-2023 10:44:44] Unable to list decisions : performing request: Get "http://localhost:8080/v1/alerts?has_active_decision=true&include_capi=false&limit=100 ": could not get jwt token: Post "http://localhost:8080/v1/watchers/login ": dial tcp [::1]:8080: connect: connection refused
After:
$ CROWDSEC_FEATURE_DISABLE_HTTP_RETRY_BACKOFF=true ./test/local/bin/cscli decisions list
FATA[11-08-2023 16:49:58] unable to retrieve decisions: performing request: Get "http://127.0.0.1:8080/v1/alerts?has_active_decision=true&include_capi=false&limit=100 ": could not get jwt token: Post "http://127.0.0.1:8080/v1/watchers/login ": dial tcp 127.0.0.1:8080: connect: connection refused
2023-08-16 21:04:07 +02:00
mmetc
afeb541eac
apic: minor refactoring ( #2415 )
...
* apic: minor refactoring
* Add whitelist length check
If user configures the file but fails to define and actual whitelist we should check length to save allocs
* Init with length from file
* extract loop method from ApplyApicWhitelists
* pass pointer
* extract loop method updateBlocklist
---------
Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-08-10 13:03:47 +02:00
Laurence Jones
93c22f29cf
Unmarshal Json ( #2414 )
...
Log the actual line that caused an error to help debugging
2023-08-09 09:42:08 +01:00
Manuel Sabban
d6361d0a40
conditional overflow doesn't overflow on capacity ( #2412 )
...
* conditional overflow doesn't overflow on capacity
* typo
2023-08-08 16:12:50 +01:00
mmetc
cd9d8f309d
CI: increase test sleep to fix flaky acquisition/file test under win ( #2410 )
...
* CI: increase test sleep to attempt fix for flaky windows acquitition/file test
* wip
2023-08-08 16:11:32 +02:00
Laurence Jones
0334a9afe8
Add method name to child logger so we can see which function is erroring when in enrichers ( #2411 )
2023-08-08 13:38:11 +01:00
Laurence Jones
a18df9c3bb
Add bouncers prune command ( #2379 )
...
* Add bouncers prune command
* No point overloading functions
* Add prune to list of commands
* change all short desc to be similar, and made it really really clear when pruning it is not recoverable
* Dont use log. and dont return error on user input to abort
2023-07-28 15:37:39 +01:00
mmetc
ffadd42779
update dependency on go-cs-lib; drop the pkg/ part ( #2393 )
2023-07-28 16:35:08 +02:00
Laurence Jones
55247cd46a
Add machines prune command ( #2011 )
...
* Add machines prune command
* Fix scope variable for naming scheme
* Add some freshness and add new features
* Fix force and fix duration if less than 60
* Allow duration to be more readable
* Fix description
* Improve func wording and make int machines length
* No point overloading functions
* Add prune to list of commands
* Check if GID is already the group if so no need to chown
* Revert "Check if GID is already the group if so no need to chown"
This reverts commit c7cef1773e
2023-07-28 15:23:47 +01:00
mmetc
ae53c0f1cc
fix "crowdsec-cli/require" log verbosity ( #2390 )
2023-07-28 09:56:20 +02:00
Thibault "bui" Koechlin
718721b341
fix a confusing debug message ( #2386 )
...
* fix a confusing debug message
* make CTIHelper simply log the error to avoid failing template rendering
2023-07-28 09:52:21 +02:00
mmetc
5cb7013575
Check cscli preconditions with crowdsec-cli/require package ( #2388 )
2023-07-27 17:02:20 +02:00
mmetc
a01ce18b98
replace imports of path with path/filepath ( #2330 )
2023-07-26 10:29:58 +02:00
Laurence Jones
389ea4293f
Add metabase version override and update ( #2370 )
...
* Add version override and update
* Ooppsie
* Quick fix
* fgs copilot
* Allow user to overwrite image, add warning for exposing metabase and general cleanup
* One ix
* Default image if not found in config, and add a warning to remove and update
* Reorder check system memory checks so it inline with @mmetc best pratices
* No need for err
* Clean up some group code
* Change ipv6 as [] seems to wildcard
* Split loopback warn and disclaimer. Add force yes to start to allow user to accept disclaimer by default
* All cmd commands are RunE clean up
* Update flag name and dont allow a shorthand
2023-07-25 14:21:25 +01:00
mmetc
395cace69f
fix double push of metrics by properly handling tickers ( #2374 )
2023-07-25 12:19:26 +02:00
blotus
7106d396dc
expose the FormatAlert function to other packages ( #2248 )
2023-07-25 09:55:39 +02:00
AlteredCoder
b52b4252c1
scenario labels to map string interface ( #2201 )
...
* labels are now map string interface
* restore api url
---------
Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-07-24 15:19:28 +02:00
mmetc
46fff0b544
Update dependency: docker/docker ( #2360 )
2023-07-24 11:53:33 +02:00
mmetc
b6b6fd026b
typo fix, uppercase 'API', adjusted log level ( #2361 )
2023-07-21 23:23:24 +02:00
Manuel Sabban
9ac5aeda79
fix the ci by adding the ability to enforce event ordering ( #2347 )
...
* fix the ci by adding the ability to enforce event ordering
2023-07-20 11:41:30 +02:00
blotus
f9ca14f010
add object key in src for S3 acquis ( #2342 )
2023-07-07 10:09:18 +02:00
blotus
1295de928a
Properly match new files on windows when doing file acquisition ( #2329 )
2023-07-06 14:45:38 +02:00
mmetc
c10bca93df
update dependencies on go-plugin and go-hclog ( #2341 )
...
* update dependencies on go-plugin and go-hclog
* bump logrus (panic fix)
* implement HCLogAdapter.Getleve() to satisfy the new interface
2023-07-06 12:01:07 +02:00
mmetc
9967d60987
errors.Wrap -> fmt.Errorf ( #2333 )
2023-07-06 10:14:45 +02:00
mmetc
17cd792826
CI: update ansible tests for re2 ( #2318 )
2023-06-29 16:35:19 +02:00
mmetc
bd41f855cf
errors.Wrap -> fmt.Errorf ( #2317 )
2023-06-29 11:34:59 +02:00
blotus
e61d5a3034
rename status to state in fire response ( #2313 )
2023-06-29 11:06:49 +02:00
mmetc
893394ef5f
rename metabase APIClient to avoid confusion ( #2305 )
2023-06-27 15:07:16 +02:00
mmetc
e404e0b608
raise error with invalid 'on_success', 'on_failure' in profile ( #2303 )
2023-06-27 15:03:07 +02:00
mmetc
85839b0199
support for stdin with "cscli decision import" and raw values ( #2291 )
...
and remove Origin from the struct, which was ignored anyway
2023-06-27 14:29:42 +02:00
mmetc
a910b7beca
non-fatal error if some datasource can't be run (i.e. journalctl but systemd is missing) ( #2309 )
...
This on the other hand, gives a new fatal error when there are no valid datasources.
In the previous version, crowdsec kept running with just a warning if no
acquisition yaml or dir were specified.
2023-06-27 10:13:13 +02:00
mmetc
507da49b5a
send metrics immediately if agents are added or removed ( #2296 )
2023-06-23 14:06:04 +02:00
mmetc
9beb5388cb
errors.Wrap -> fmt.Errorf; clean up imports ( #2301 )
2023-06-23 14:04:58 +02:00
mmetc
e42841cd00
Change api_key encoding to base64 to comply with bcrypt max size ( #2302 )
2023-06-23 13:54:36 +02:00
mmetc
62caffb102
update leakybucket readme ( #2298 )
2023-06-22 15:35:01 +02:00
mmetc
fddf597040
errors.Wrap -> fmt.Errorf; clean up imports ( #2297 )
2023-06-22 15:01:34 +02:00
mmetc
8bfeb7d90d
Update go dependencies ( #2293 )
...
- update fatih/color (fix windows issue)
- update mongo-driver (fix build issue)
- go.mod: merge two "require" blocks
- update semver dependency (same version as indirect dep), fix test checks in cscli setup
- remove gotest.tools dependency (use testify, cstest)
- update x/ exp, mod, sys dependencies
2023-06-22 11:31:41 +02:00
Emanuel Seemann
40e6b205bc
Add bayesian bucket type ( #2290 )
2023-06-21 15:08:27 +02:00
mmetc
da6106bd23
spellcheck/style leakybucket readme ( #2294 )
2023-06-21 11:47:07 +02:00
mmetc
f7409d47be
fix error message when failing to parse ip address ( #2292 )
...
Co-authored-by: Thibault "bui" Koechlin <thibault@crowdsec.net>
2023-06-21 09:22:25 +02:00
Laurence Jones
2c8769adf6
Update jsonextract.go ( #2287 )
...
Return nil instead of empty string as ParseKV does the same
2023-06-16 18:34:55 +01:00
mmetc
b9a3acb03f
light pkg/parser cleanup ( #2279 )
...
* pkg/parser: clean up imports
* remove duplicate import
* simplify boolean expression
* don't check length before range
* if..else if.. -> switch/case
* errors.Wrap -> fmt.Errorf
* typo, lint
* redundant break
2023-06-13 13:16:13 +02:00
mmetc
76429f033a
trim pkg/types: move DataSet/GetData to pkg/cwhub, removed unused Clone function ( #2271 )
2023-06-08 16:49:51 +02:00
mmetc
cf747d65e0
fix missing import ( #2275 )
2023-06-08 15:49:37 +02:00
mmetc
25bb23d8b7
minor refactor to pkg/types, cscli machines ( #2270 )
...
* cleanup: separate ui and logic
* trim some code from pkg/types
2023-06-08 15:08:51 +02:00
mmetc
6096cb3c9b
Move grok_pattern.go away from pkg/types to trim bouncer dependencies ( #2269 )
2023-06-08 15:07:30 +02:00
mmetc
8da9d5eefd
don't log notification error if not running under systemd ( #2274 )
2023-06-08 15:04:48 +02:00
mmetc
5b3200173e
don't pre-create log files (not required anymore) ( #2267 )
...
The lumberjack package fixed the issue in natefinch/lumberjack#83 (tested with umask 002) and this code is now redundant since we updated the dependency to v2.2.1.
2023-06-07 12:58:35 +02:00
mmetc
edd062522d
build against libre2-dev if found ( #2255 )
2023-06-06 15:46:25 +02:00
mmetc
3cc6b2c0d0
CI: add tests for metrics configuration ( #2251 )
2023-06-05 23:17:30 +02:00
mmetc
0191faf3a8
update notif threshold test on windows ( #2265 )
2023-06-05 22:58:13 +02:00
mmetc
e3cb4ab2c4
do not send more than group_threshold alerts at once to a notification plugin ( #2264 )
...
* do not send more than group_threshold alerts at once to a notification plugin
* Use generic Chunks function, updated tests
---------
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-06-05 12:55:03 +02:00
mmetc
a4eee41fd7
log.Warning if a notification is configured twice ( #2240 )
2023-06-02 14:41:50 +02:00
mmetc
396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap ( #2235 )
...
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
2023-06-01 16:31:56 +02:00
mmetc
12c32d507c
CI: refactoring pkg/csplugin tests ( #2247 )
2023-06-01 10:33:08 +02:00
mmetc
92a9d6c321
types.InSlice() -> slices.Contains() ( #2246 )
2023-05-31 12:39:22 +02:00
Laurence Jones
4fbc3402fb
Update KV ignore whitespace before and after = ( #2236 )
...
* Update KV ignore whitespace before and after `=`
* Update helpers.go
Don't need whitespace infront of KEY
* Add some tests to ensure edge cases
* Ensure quoted and unquoted values act the same
2023-05-26 15:35:46 +01:00
blotus
6720d89845
fix lock when dumping the parsing state in explain mode ( #2234 )
2023-05-26 15:23:50 +01:00
blotus
f6924f8c57
generate asserts for evt.Unmarshaled in hubtest ( #2214 )
2023-05-26 11:44:58 +02:00
mmetc
9167bd107d
decouple bouncer dependencies: use go-cs-lib/pkg/ptr ( #2228 )
2023-05-25 15:43:39 +02:00
mmetc
b2d3520519
decouple bouncer dependencies: use go-cs-lib in test code ( #2229 )
2023-05-25 15:37:44 +02:00
mmetc
364b833d67
test cleanup: remove /tmp/crowdsec_tests* directories ( #2232 )
2023-05-25 15:32:32 +02:00
Laurence Jones
0416a41d58
Log info capi whitelists ( #2220 )
...
* add infof command if err was nil
* Fix golint
* Make message more readable and log individual stats
* Missed a d
* Remove '
* simplify if/else logic
---------
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-05-25 10:28:08 +01:00
mmetc
025f14f879
merge system cert pool with own certs ( #2226 )
2023-05-25 10:10:58 +02:00
mmetc
e5fe74ce77
decouple bouncer dependencies: use go-cs-lib/pkg/ptr in apiclient ( #2227 )
2023-05-25 10:08:52 +02:00
mmetc
534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* ( #2216 )
...
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
blotus
6e3ca35941
fallback to master for hub index download if it does not exist ( #2210 )
2023-05-17 11:20:53 +02:00
blotus
412b4c4b0b
fix incorrect version strip ( #2206 )
2023-05-17 01:13:55 +02:00
Thibault "bui" Koechlin
77f2968267
fix the behavior of json unmarshal to not return the full map ( #2199 )
2023-05-16 09:10:38 +02:00
Laurence Jones
424215f228
Add ParseKV helper and rework UnmarshalJSON as a proper helper ( #2184 )
2023-05-12 09:43:01 +02:00
mmetc
e1f5ed41df
Implement "cscli config show-yaml" ( #2191 )
2023-05-11 21:01:13 +02:00
blotus
4ae41a363d
add Hostname helper in expr and templating ( #2193 )
2023-05-11 14:25:04 +02:00
blotus
71b7a594bd
add indexes on the FK between alerts and {decisions,metas,events} ( #2188 )
2023-05-11 13:49:01 +02:00
blotus
2701454f23
defaults to inotify to detect changes in file datasource to avoid too many call to stat() ( #2181 )
2023-05-09 10:03:55 +02:00
blotus
e1f4a71357
readd KeyExists expr helper ( #2180 )
2023-05-04 16:55:34 +02:00
blotus
a753ea6981
Add B64decode expr helper ( #2183 )
2023-05-04 14:15:20 +02:00
Thibault "bui" Koechlin
8f71edaadd
do not error on this filter ( #2182 )
2023-05-04 13:06:15 +02:00
Thibault "bui" Koechlin
4ff8f498ce
add a LogInfo expr helper ( #2179 )
2023-05-03 10:07:11 +02:00
AlteredCoder
6bb20fa951
fix issue #2172 ( #2177 )
2023-04-28 16:32:46 +02:00
AlteredCoder
c0e6c1ac78
Fix chooseHubBranch when latest() doesn't work ( #2178 )
...
* Fix chooseHubBranch when latest() doesn't works
2023-04-28 11:24:04 +02:00
Thibault "bui" Koechlin
3041023ed8
add an optional flag to disable the fetch ( #2169 )
2023-04-14 11:39:16 +02:00
Thibault "bui" Koechlin
66dfded0cf
significantly increase the max number of scenarios to be sent ( #2170 )
2023-04-14 11:39:07 +02:00
mmetc
0c5d233563
Minor cleanup and dead code removal ( #2166 )
2023-04-12 16:57:38 +02:00
Laurence Jones
9a5a937695
Make it more obvious that parser succeeded but was whitelisted ( #2167 )
...
* Make it more obvious that parser succeeded but was whitelisted
* Add more verbose by placing whitelist reason next to why it is ignored
2023-04-12 10:48:42 +01:00
blotus
0279e549bd
check if the acquis tomb is dying while processing logs in replay mode for file/s3/docker ( #2152 )
2023-04-04 13:57:06 +02:00
mmetc
3132aa54b7
Properly load k8s audit configuration ( #2158 )
2023-04-03 21:55:31 +02:00
mmetc
38ab6be7c2
Allow feature.yml to change available subcommands ( #2156 )
2023-04-03 10:11:56 +02:00
mmetc
3fa555fb25
Rename k8s_audit to k8s-audit (easier to type, consistent with labels) ( #2153 )
2023-04-03 09:53:38 +02:00
blotus
61bea26486
Add transform configuration option for acquisition ( #2144 )
2023-03-29 16:04:17 +02:00
blotus
772d5b5c32
Add experimental support for re2 ( #2138 )
2023-03-28 16:26:47 +02:00
blotus
1095f6c875
use expr.Function for custom functions instead of passing them in the env ( #2133 )
2023-03-28 10:49:01 +02:00
Thibault "bui" Koechlin
169b844212
fix awkward stacktrace in conditional filter ( #2145 )
2023-03-27 16:01:42 +02:00
mmetc
d769fff1e8
File acquisition: log "file reopen" events instead of writing to stderr ( #2139 )
2023-03-24 11:24:36 +01:00
mmetc
3884c5f47d
Unit tests: remove leftover files ( #2134 )
2023-03-22 13:51:37 +01:00
Thibault "bui" Koechlin
a3e5f0a3a0
fix dateparse ( #2135 )
2023-03-22 08:20:21 +01:00
blotus
91eb39cff6
New PAPI commands: reauth + force_pull ( #2129 )
2023-03-21 14:06:19 +01:00
blotus
dc38e5ac00
S3 acquisition datasource ( #2130 )
2023-03-21 13:54:52 +01:00
Thibault "bui" Koechlin
a74e424d53
support ip and cidr based whitelists for capi and 3rd party blocklists ( #2132 )
...
* support ip and cidr based whitelists for capi and 3rd party blocklist
2023-03-21 11:50:10 +01:00
Thibault "bui" Koechlin
d87f088b8f
match expr helper ( #2126 )
...
* match expr helper
2023-03-21 10:39:17 +01:00
Thibault "bui" Koechlin
618be9ff68
properly update the time structure within event ( #2122 )
...
* properly update the time structure within event to ensure it works in time-machine
* move LIVE and TIMEMACHINE to pkg/types : less code needs to import leakybucket package, and we avoid duplicating constants
2023-03-16 16:25:50 +01:00
blotus
c77fe16943
actually fix expr-debugger to work with the new version ( #2124 )
2023-03-16 15:20:48 +01:00
blotus
94c7efdb5b
add ToString() helper ( #2100 )
2023-03-16 15:20:31 +01:00
blotus
b1f2063a9a
Only support pgx driver for postgresql ( #2118 )
2023-03-16 11:02:31 +01:00
Thibault "bui" Koechlin
855f9e6f8d
protect map w/ mutex to avoid concurrent map writes with cscli explain when having many concurrent parser routines ( #2113 )
2023-03-16 11:01:25 +01:00
Manuel Sabban
b451d190b7
try to make reproducible build work ( #2119 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2023-03-13 17:26:33 +01:00
blotus
6aaf3cd50b
Update expr to 1.12.2 ( #2110 )
2023-03-09 16:56:11 +01:00
mmetc
e161507d08
Lint (type inference): remove redundant type declarations ( #2111 )
2023-03-09 11:56:02 +01:00
Thibault "bui" Koechlin
d95b7afe61
Distance support : Impossible travel ( #2108 )
...
* add distance helpers
2023-03-08 18:29:42 +01:00
Thibault "bui" Koechlin
9d5aaf5ea2
add --origin to cscli decisions delete ( #2109 )
2023-03-08 18:29:20 +01:00
Thibault "bui" Koechlin
5b0fe4b7f1
support for regexps result cache ( #2104 )
...
* support for regexps result cache : gcache + xxhash
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-03-08 16:07:49 +01:00
blotus
16a3be49e2
do not try to load PAPI is url is not set ( #2099 )
2023-03-06 15:38:58 +01:00
blotus
85ab9c68a2
Add cscli papi status and cscli papi sync ( #2091 )
2023-03-03 13:46:28 +01:00
mmetc
f6d6c5bb2b
Add tests and typo fixes ( #2092 )
2023-03-03 11:06:27 +01:00
AlteredCoder
01ea78c10e
Strip version with ~ instead of - ( #2076 )
2023-02-25 20:05:48 +01:00
Laurence Jones
75d8b821ff
Explain successful parsers only ( #2063 )
...
* Add option to filter down explain to successful parsers useful for me who has every collection installed
* Altered naming conventions so it makes more sense when reading
2023-02-24 13:49:17 +00:00
Laurence Jones
8acce4637a
Option to disable remote lapi registration ( #2010 )
...
* Allow to disable remote lapi registration
* Extract method and make it extendable as a generic middleware
* Change method name so it make sense to read abort remote if <config>
* golint
2023-02-24 13:44:21 +00:00
mmetc
20a1bc7d44
chore: simplify pkg/database/alerts ( #2062 )
2023-02-23 10:25:01 +01:00
mmetc
be18fea136
Propagate taints to top collections ( fix #2064 ) ( #2066 )
2023-02-21 22:12:08 +01:00
mmetc
76ea3a063f
fix message "empty scenario"
2023-02-21 09:59:56 +01:00
blotus
90c38db9f2
Stream decisions from db ( #1927 )
2023-02-20 15:26:30 +01:00
JDEV
12a4a5fb14
CAPI error code handling tests ( #2027 )
...
* Registration mocked error cases
* Authentication mock error cases
* mini facto
* check that getMEtric still has bouncers/machines keys in output even with empty collections
* fixed defer body close(), no need to defer and fprint arg
* fix fatal call
---------
Co-authored-by: jdv <julien@crowdsec.net>
2023-02-17 14:57:46 +01:00
blotus
83c3818504
Do not try to refresh JWT token when doing a login request ( #2059 )
2023-02-16 16:16:26 +01:00
Laurence Jones
5aca11af70
Show s00 stats instead of "first_parser" ( #2055 )
...
* show s00 if verbose is provided
* Clean up code
* Fix failing test
2023-02-14 14:36:08 +00:00
Cristian Nitescu
ecb32d74c6
optimize blocklist fetch ( #2039 )
2023-02-13 15:06:14 +01:00
Cristian Nitescu
f280505eaa
omtimization - remove useless login call ( #2036 )
2023-02-13 15:05:58 +01:00
blotus
812b87ab48
Add IsIPV4() and IsIP() helpers ( #2050 )
2023-02-10 14:44:42 +01:00
Thibault "bui" Koechlin
0f5560b62a
more strings helpers ( #2040 )
...
* more strings helpers
2023-02-09 15:23:21 +01:00
Thibault "bui" Koechlin
1d7d377f8b
changes following BL tests ( #2038 )
...
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-02-08 10:35:21 +01:00
Thibault "bui" Koechlin
a0b264047c
allow user to specify stash's cache strategy ( #2037 )
2023-02-06 15:42:55 +01:00
Cristian Nitescu
987f119c4b
v3 capi and blocklists links support ( #2019 )
...
* v3 model generation
* v3 model generation
* comms
* fixes after master merge
* missing reader close
* use constants defined for types
---------
Co-authored-by: bui <thibault@crowdsec.net>
2023-02-06 14:06:14 +01:00
mmetc
b6be18ca65
cscli setup ( #1923 )
...
Detect running services and generate acquisition configuration
2023-02-06 07:33:04 +01:00
AlteredCoder
7e871d2278
rename PAPI base URL ( #2033 )
2023-02-03 12:10:02 +01:00
Thibault "bui" Koechlin
e927717fa0
Polling API Integration ( #1715 )
...
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-31 14:47:44 +01:00
mmetc
e37d09e5b4
use helpers for shorter tests, add a couple of error cases ( #2016 )
2023-01-26 17:13:31 +01:00
mmetc
3fb3decf49
error if tls.key_file or cert_file are missing ( #2020 )
2023-01-26 17:12:59 +01:00
mmetc
02be5f3618
allow literal $ in plugin configuration ( #2015 )
2023-01-23 16:28:43 +01:00
mmetc
47cc60bda9
allow use of literal $ in config.yaml ( #2012 )
2023-01-23 10:29:29 +01:00
mmetc
e5833699c0
cscli config feature-flags ( #2006 )
2023-01-20 09:32:10 +01:00
Thibault "bui" Koechlin
4f29ce2ee7
CTI API Helpers in expr ( #1851 )
...
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-19 08:45:50 +01:00
Marco Mariani
0c35d9d43c
wip
2023-01-18 15:15:18 +01:00
Marco Mariani
4f25738d6b
wip
2023-01-18 15:15:18 +01:00
Marco Mariani
47dbfa770d
configure logging earlier
2023-01-18 15:15:18 +01:00
Marco Mariani
91b0f8fee1
load custom configuration paths when agent is disabled
2023-01-18 15:15:18 +01:00
Marco Mariani
2e91a82aa7
load feature.yaml as soon as possible
2023-01-18 15:15:18 +01:00
Thibault "bui" Koechlin
f25fdecc3f
normalize scopes for alerts and decisions ( #2001 )
...
* normalize scopes for alerts and decisions
2023-01-18 14:50:03 +01:00
mmetc
51800132cd
improve feature flag logging ( #1986 )
...
For cscli: it should provide a terse output, not nag users with configuration details. Although it's usually important that cscli and crowdsec have the same enabled features, having it list them every time the command is invoked can be too much.
For crowdsec: when features are set from the environment, it's too early to log where we should. So we can use log.Debug at activation time, and list them again once logging is configured.
- wrap some functions in csconfig for convenience and DRY
- for each enabled feature, log.Debug
- log all enabled features once as Info (crowdsec) or Debug (cscli)
- file does not exist -> log.Trace
2023-01-13 13:42:42 +01:00
Cristian Nitescu
73663ff9e7
log the request error even in case of retry ( #1988 )
2023-01-13 12:58:12 +01:00
mmetc
ba4396e52c
fix flaky parser unit test ( #1985 )
2023-01-12 17:03:25 +01:00
Thibault "bui" Koechlin
6fb962a941
Allow parsers to capture data for future enrichment ( #1969 )
...
* Allow parsers to capture data in a cache, that can be later accessed via expr helpers (fake multi-line support)
2023-01-11 15:01:02 +01:00
mmetc
cd4dabde0e
silence yaml.local explicitly in cscli, keep in crowdsec/bouncer logs ( #1981 )
2023-01-11 09:50:46 +01:00
Laurence Jones
ca12432a2a
Change patch to debug, if user has a local overide they will get informed every cscli call ( #1980 )
2023-01-10 10:05:18 +00:00
Cristian Nitescu
7284c0a47a
retry with backoff requests to CAPI ( #1957 )
...
* backoff on refresh token error
* fix tls communication with lapi and user/pw auth (#1956 )
allow self-signed TLS encryption with user/pw auth
docker:
- remove defaults for certificate file locations
- new envvar INSECURE_SKIP_VERIFY
- register agent before TLS settings (cscli machine add removes them
from the credentials file)
* separate cscli cobra constructors: lapi, machines, bouncers, postoverflows (#1945 )
* use feature toggling to improve testability with http retry backoff
* Add parse unix to dateparse enricher (#1958 )
Add parse unix is we do have a strTime but wasnt parsed using convential golang time
* func tests: redirect stderr to filter extra logs (#1961 )
* backoff on refresh token error
* use feature toggling to improve testability with http retry backoff
* refactor feature backoff toggle for tests
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-01-09 14:49:21 +01:00
blotus
a84e4b6b15
Add conditional bucket ( #1962 )
2023-01-06 09:26:16 +01:00
AlteredCoder
185f9ad541
Alert context ( #1895 )
...
Co-authored-by: bui <thibault@crowdsec.net>
2023-01-04 16:50:02 +01:00
mmetc
033082a31e
ParseUnix() test fix: force UTC ( #1970 )
2023-01-04 16:22:17 +01:00
mmetc
2d81e751a1
fix parser test 2k23 ( #1971 )
2023-01-04 15:46:16 +01:00
Laurence Jones
fd1c38811e
Add parse unix to dateparse enricher ( #1958 )
...
Add parse unix is we do have a strTime but wasnt parsed using convential golang time
2022-12-30 12:47:14 +00:00
mmetc
72c1753fb7
fix tls communication with lapi and user/pw auth ( #1956 )
...
allow self-signed TLS encryption with user/pw auth
docker:
- remove defaults for certificate file locations
- new envvar INSECURE_SKIP_VERIFY
- register agent before TLS settings (cscli machine add removes them
from the credentials file)
2022-12-29 22:00:11 +01:00
Laurence Jones
401739b036
Add unix expr helper ( #1952 )
...
* Add unix expr helper
* Add original value not parsed error
* return early if cannot parse
* Add tests
* Fix negative value
2022-12-29 14:53:06 +00:00
Thibault "bui" Koechlin
e4463c412b
Improve warnings around lack of evt.StrTime field ( #1954 )
...
* fix #1951 : improve error messages
* make hubtest warn you if you're missing evt.StrTime in your logs
2022-12-29 15:03:32 +01:00
mmetc
6efc2688b1
simplify feature flags ( #1947 )
...
Now checking for a feature flag is a one liner,
with no need to control errors.
if fflag.Crowdsec.CscliSetup.IsEnabled() {
...
}
2022-12-26 14:23:41 +01:00
mmetc
5d2c99bb17
runtime feature flag initialization
2022-12-21 17:19:20 +01:00
mmetc
ff88faf402
updated localstack dependencies, added build cache
2022-12-21 12:20:01 +01:00
mmetc
a32aa96752
feature flags ( #1933 )
...
Package fflag provides a simple feature flag system.
Feature names are lowercase and can only contain letters, numbers, undercores
and dots.
good: "foo", "foo_bar", "foo.bar"
bad: "Foo", "foo-bar"
A feature flag can be enabled by the user with an environment variable
or by adding it to {ConfigDir}/feature.yaml
I.e. CROWDSEC_FEATURE_FOO_BAR=true
or in feature.yaml:
```
---
- foo_bar
```
If the variable is set to false, the feature can still be enabled
in feature.yaml. Features cannot be disabled in the file.
A feature flag can be deprecated or retired. A deprecated feature flag is
still accepted but a warning is logged. A retired feature flag is ignored
and an error is logged.
A specific deprecation message is used to inform the user of the behavior
that has been decided when the flag is/was finally retired.
2022-12-20 16:11:51 +01:00
he2ss
579cecde04
apiclient: fix http roundtrip (clone body also) ( #1758 )
...
* apiclient: fix http roundtrip (clone body also)
2022-12-14 16:42:46 +01:00
Laurence Jones
fe23da6e0c
Add postgres socket support, clean some code ( #1926 )
2022-12-12 16:08:19 +00:00
Laurence Jones
11965f08db
Add socket support to mysql ( #1911 )
2022-12-08 09:33:08 +00:00
mmetc
cc228f1868
Typos, grammar ( #1905 )
2022-12-06 15:55:27 +01:00
blotus
fdda940ac0
Add Kubernetes audit acquisition ( #1767 )
2022-12-06 13:47:29 +01:00
mmetc
fd3e668fe1
add -error flag to crowdsec binary ( #1903 )
2022-12-03 08:56:11 +01:00
mmetc
fa0e590778
removed pid_dir ( #1906 )
2022-12-02 13:42:43 +01:00
mmetc
4a6a9c4355
acquisition: validate datasources before configuration (static checks) ( #1841 )
...
* acquisition: validate datasources before configuration (allow static configuration checks)
* remove comment
* import reviser, format
* error wrap
2022-11-30 17:36:56 +01:00
blotus
60f1228030
use a copy of bucket processors in LeakRoutine ( #1902 )
2022-11-30 10:59:47 +01:00
mmetc
104f5d1fe6
lint: error handling cleanup ( #1877 )
2022-11-29 09:16:07 +01:00
mmetc
66543493b5
fix nil dereference: check that httpServer is set before shutting down ( #1893 )
2022-11-28 11:55:08 +01:00
mmetc
fde9640364
Docker refactoring, tls setup ( #1869 )
2022-11-28 10:35:12 +01:00
blotus
c5079ac15e
invalidate agent token on 403 as well ( #1888 )
2022-11-25 14:35:50 +01:00
mmetc
5bdd3bbfcb
require at least go 1.18 to build ( #1884 )
2022-11-24 11:29:54 +01:00
Laurence Jones
4ac01ed880
Update perms for group read ( #1876 )
2022-11-21 09:49:56 +00:00
mmetc
3beb84bcfe
print missing "AS" values as empty strings instead of "0 " ( #1867 )
2022-11-14 09:55:53 +01:00
Thibault "bui" Koechlin
523343b174
notify when community-blocklist starts pull ( #1845 )
...
* minor change to notify blocklist pull update, will make eventual troubleshooting easier
2022-11-08 10:44:25 +01:00
Thibault "bui" Koechlin
3b4da7e637
fix #1860 : Only repeat the WAL warning once ( #1863 )
...
* fix #1860
2022-11-07 16:36:39 +01:00
mmetc
895691dad1
enabled linters: gocritic, nilerr ( #1853 )
2022-11-07 10:36:50 +01:00
Manuel Sabban
8aca00326d
fix ticker ( #1858 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-11-04 13:56:43 +01:00
Laurence Jones
668627f890
Add error checking to lookup host ( #1847 )
2022-10-31 18:38:01 +00:00
mmetc
344b1dc559
fixed package tests w/wal, gitignore/typos ( #1849 )
2022-10-31 10:02:51 +01:00
mmetc
df88f4e1e9
randomize pull, push and metric intervals; reload crowdsec only when hub changed ( #1846 )
2022-10-28 13:55:59 +02:00
mmetc
02d2eab18c
update golangci-lint to 1.50 and fixes ( #1828 )
2022-10-26 15:11:37 +02:00
ThinkChaos
22479a289d
Add LookupHost expr lib func ( #1775 )
2022-10-26 10:17:48 +01:00
mmetc
2088bb1f91
fix for #1839 ( #1840 )
2022-10-26 11:02:12 +02:00
blotus
b7c4bfd4e3
Use explicit transaction when inserting community blocklist ( #1835 )
2022-10-26 10:48:17 +02:00
mmetc
e545933923
fix(cscli): correct and test the behavior of "cscli collections delete" ( #1824 )
2022-10-25 14:10:51 +02:00
blotus
bb2f0e938f
Blocklist: Do not duplicate decisions when pulling ( #1796 )
2022-10-19 15:51:40 +02:00
Thibault "bui" Koechlin
ae6bf39495
support decisions deletion via scenario + alerts delete via ID ( #1798 )
2022-10-19 14:37:27 +02:00
mmetc
6b0097a24b
change warning to debug when directories are missing in hub sync ( #1819 )
2022-10-18 10:32:54 +02:00
mmetc
2b7e3ff1e7
warn if no acquisition files are found, acquisition_test refactoring, tests ( #1816 )
2022-10-17 17:32:08 +02:00
mmetc
ec0d2a5ed2
refactor broker_test.go, extract cstest/filenotfound*.go ( #1815 )
2022-10-17 14:17:23 +02:00
mmetc
a96b3e077d
rename pkg/cstest -> pkg/hubtest ( #1811 )
...
keep cstest for generic helper functions
this also avoids circular imports in test files
2022-10-17 09:24:07 +02:00
mmetc
8fecc2c00b
enable staticcheck linter; fixes ( #1806 )
...
- explicitly ignore returned parameters
- replace Walk with faster WalkDir
- log path error during hub dir sync
- colorize static unit tests
- removed duplicate import in crowdsec/main.go
- typos
- func tests: default datasource in tests/var/log instead of /tmp
- action setup-go v3
2022-10-14 16:12:21 +02:00
Manuel Sabban
7359586f1c
fix ticker mix up ( #1807 )
...
Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2022-10-13 14:30:27 +02:00
