Commit graph

226 commits

Author SHA1 Message Date
alteredCoder
04f7dbd1f5 update 2021-10-26 15:13:36 +02:00
alteredCoder
908b04028e push csconfig console 2021-10-26 15:13:36 +02:00
alteredCoder
1372c49505 fix the swagger model 2021-10-26 15:13:36 +02:00
alteredCoder
03dbce1e50 add structure 2021-10-26 15:13:36 +02:00
alteredCoder
2c6a279b9b add load configuration file 2021-10-26 15:13:36 +02:00
alteredCoder
52edfba2b3 add console type 2021-10-26 15:13:36 +02:00
blotus
25a2d528b0
Alerts flush: Optimization of the flush mechanism (batch and limit to one job) + add cscli alerts flush command (#1024)
- Don't allow running more than one alert flush job at a time to prevent runaway CPU usage in some case. (fix High CPU after Upgrade to 1.2.0 #1022)
 - Add a cscli alerts flush command to manually flush the alerts in the database (fixes Improvement/Manual flush mechanism #1023 ).
 - Enable cascading deletion on alerts as we upgraded ent: Deleting an alert in the database will automatically delete all related decisions, events and meta
 - Add an index on alerts.id to try to improve flush performance with very big sqlite database.
- Flush alert now operates in batch
2021-10-26 13:33:45 +02:00
Thibault "bui" Koechlin
3f99330b3d
Entgo 0.9 (#1018)
* update entgo & sqlite to latest version

* schema update
2021-10-22 16:15:57 +02:00
Shivam Sandbhor
a7b1c02bd5
Fix bugs in cloudwatch acq (#991)
* Fix bugs in cloudwatch acq

- Fix concurrent writes to map streamIndexes
- Fix multiple cases of modifying while iterating on slice.
- Fix order of fetching cloudwatch events.
- Remove `startup` hack.

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>

* Fix cloudwatch tests

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-10-22 10:35:05 +02:00
Thibault "bui" Koechlin
3bb2128bf4
fix sort :/ (#1007) 2021-10-12 19:16:24 +02:00
Thibault "bui" Koechlin
1bd6b8f7b9
Multiple fixes (#1006)
* fix #1005 : timestamp in trigger timemachine buckets

* attempt at consistent bucket order for hubtest
2021-10-12 14:09:17 +02:00
Thibault "bui" Koechlin
2961a0ed02
ensure machineID is included early enough into the alert (#1004) 2021-10-11 15:02:16 +02:00
blotus
2bc9f33e12
add ParseUri() expr helper (#994) 2021-10-08 16:50:31 +02:00
AlteredCoder
0ccc69696b
Break on success when alert already has decision (#997) (#999)
* Break on success when alert already has decision (#997)

Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-10-05 11:30:34 +02:00
Thibault "bui" Koechlin
af4bb350c0
hubtests revamp + cscli explain (#988)
* New hubtest CI for scenarios/parsers from the hub
 * New `cscli explain` command to visualize parsers/scenarios pipeline

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Cristian Nitescu <cristian@crowdsec.net>
2021-10-04 17:14:52 +02:00
Thibault "bui" Koechlin
c2fd173d1e
fix node success logic (#993)
* fix node success logic : only fail node on child failure if mother node has no successfull grok
2021-09-28 17:58:07 +02:00
he2ss
fb308d5596
fix plugins logging in right level (#990) 2021-09-28 14:44:21 +02:00
he2ss
db5ffb0040
Update test env (#987)
* update test_env
2021-09-24 18:06:30 +02:00
blotus
f0db3742de
fix usage of regex.Match in cloudwatch module (#986) 2021-09-23 13:52:05 +02:00
Shivam Sandbhor
cca76da2d6
Fix crash if plugin config is broken (#964)
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-09-10 14:25:34 +02:00
he2ss
e651379964
add jsonExtractUnescape Helper (#962)
* add jsonExtractUnescape Helper
2021-09-10 12:43:11 +02:00
AlteredCoder
5ae69aa293
fix stacktrace when mmdb file are not present (#935)
* fix stacktrace when mmdb file are not present
2021-09-09 16:27:30 +02:00
blotus
7a1b955ad1
use our fork of grokky (#953) 2021-09-09 14:46:16 +02:00
Shivam Sandbhor
b8e24a1e0b
Make plugin runner configurable and run only registered plugins (#944)
* Make plugin runner configurable and run only registered plugins
2021-09-08 11:36:42 +02:00
Thibault "bui" Koechlin
0ad6165ed2
fix release drafter + readme + remove dead readme for acquis (#933) 2021-09-03 09:07:24 +02:00
Manuel Sabban
d7d591ff84
update to use cdn for hub (#920)
* update to use cdn for hub
* add cdn for version
* fix unit tests accodingly with new cdn

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2021-09-02 15:17:37 +02:00
Thibault "bui" Koechlin
bed90a832e
fix #919 : display error message (#929)
* fix #919

* fix tests
2021-09-02 12:46:32 +02:00
Thibault "bui" Koechlin
589cb72d41
enforce a bit more parsing for resillience (#928) 2021-09-02 12:34:20 +02:00
Shivam Sandbhor
b40fd36607
Add plugin interface code in protobufs package (#921)
Signed-off-by: Shivam Sandbhor <shivam.sandbhor@gmail.com>
2021-08-31 14:40:17 +02:00
Thibault "bui" Koechlin
68c11dd827
don't try to send/don't notify if plugin chan is nil (#923) 2021-08-31 14:39:32 +02:00
blotus
b5d0d56a11
add support for --since in journalctl DSN (#917) 2021-08-31 12:40:22 +02:00
ThinkChaos
448a227079
Minor changes to specific logs (#900)
- Minor changes to specific logs
- Fix LAPI to not push signals to CAPI when disabled #907
2021-08-25 18:30:05 +02:00
Thibault "bui" Koechlin
c188d401a3
Improve CAPI pull management (#871)
* prepare for new consensus : thousands of ips

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2021-08-25 11:45:29 +02:00
Thibault "bui" Koechlin
950759f6d6
Output plugins (#878)
* Add plugin system for notifications (#857)
2021-08-25 11:43:29 +02:00
Manuel Sabban
4dbbd4b3c4
Download datafile (#895)
* add the ability to download datafile on cscli hub upgrade on files are missing
* fix stuff + lint
* fix error management

Co-authored-by: sabban <15465465+sabban@users.noreply.github.com>
2021-08-19 09:08:20 +02:00
Shivam Sandbhor
f64f20fd53
Document scope parameter for stream API (#897)
Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>
2021-08-18 16:05:56 +02:00
Nanik
b0746fbc4d
fix: add /health endpoint (#881)
* fix: add /health endpoint
2021-08-18 09:06:01 +02:00
Thibault "bui" Koechlin
05ac3ca402
if profile is in debug, log debug even if it matched the profile (#894) 2021-08-17 16:50:16 +02:00
Thibault "bui" Koechlin
25ed1c265d
fix #885 : remove dead dependencies for plugin (#891) 2021-08-17 10:32:15 +02:00
Thibault "bui" Koechlin
fc7369c4ea
Fix big serialized entries (#877)
* bump serialized to 8k

* handle oversized serialized entry : progressively strip its size down
2021-08-03 15:46:10 +02:00
Thibault "bui" Koechlin
01028d0a09
Goroutine leak hunt (#874)
* close the writers of gin loggers + kill the tomb of httpServer

* body close defer
2021-07-30 11:41:17 +02:00
blotus
cedfca07c2
don't wait for acquis tomb if we have no sources (#868) 2021-07-28 08:58:44 +02:00
Thibault "bui" Koechlin
b6ee006078
ensure decisions from CAPI have proper case (#848) 2021-07-02 11:23:46 +02:00
Thibault "bui" Koechlin
033c8e17e8
fix #842 #837 (#845)
* fix #842 and move preflight checks tgth

* handle new container name

Co-authored-by: AlteredCoder <AlteredCoder>
2021-07-01 18:15:22 +02:00
blotus
3994aec7fe
add console enroll command to cscli (#828) 2021-06-28 17:34:19 +02:00
Thibault "bui" Koechlin
7f0cac8ee6
add support for 'expression' (fix #822) in grok patterns (#830)
* add support for 'expression' (fix #822) in grok patterns

* add tests
2021-06-21 09:07:33 +02:00
Thibault "bui" Koechlin
ce6a61df1c
Refactor Acquisition Interface (#773)
* Add new acquisition interface + new modules (cloudwatch, syslog)

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2021-06-11 09:53:53 +02:00
Thibault "bui" Koechlin
71c1d9431f
fix #823 : lower JsonExtract debug (#824)
* lower key not found log level, fix #823
2021-06-02 14:27:34 +02:00
Shivam Sandbhor
f25d02a7c8
Allow bouncers to filter decisions by scope (#817)
Signed-off-by: Shivam Sandbhor <shivam@crowdsec.net>
2021-05-31 15:07:09 +02:00
Thibault "bui" Koechlin
bf6b791420
fix #781 - avoid unconsistent body : do not send NbDeleted on error (#812) 2021-05-28 11:17:30 +02:00