Commit graph

112 commits

Author SHA1 Message Date
Thibault "bui" Koechlin
19d36c0fb2
Support console options in console enroll (#2760)
* make dev.yaml has a valid/default console path

* simplify and make more consistent help message about console opts

* allow enroll to specify options to enable

* allow 'all' shortcut for --enable
2024-01-19 15:49:00 +01:00
mmetc
24b5e8f100
Fix #2733 "cscli hang forever when i try to delete a decision" (#2745) 2024-01-16 09:16:21 +01:00
mmetc
fca8883cd9
cscli capi status -> message for missing credentials (#2730)
* cscli capi status -> message for missing credentials
* lint
2024-01-12 14:41:36 +01:00
mmetc
6960419a2e
Remove redundant file check for capi_whitelists_path (#2728) 2024-01-12 14:17:01 +01:00
mmetc
89f704ef18
light pkg/api{client,server} refact (#2659)
* tests: don't run crowdsec if not necessary
* make listen_uri report the random port number when 0 is requested
* move apiserver.getTLSAuthType() -> csconfig.TLSCfg.GetAuthType()
* move apiserver.isEnrolled() -> apiclient.ApiClient.IsEnrolled()
* extract function apiserver.recoverFromPanic()
* simplify and move APIServer.GetTLSConfig() -> TLSCfg.GetTLSConfig()
* moved TLSCfg type to csconfig/tls.go
* APIServer.InitController(): early return / happy path
* extract function apiserver.newGinLogger()
* lapi tests
* update unit test
* lint (testify)
* lint (whitespace, variable names)
* update docker tests
2023-12-14 14:54:11 +01:00
blotus
04f3dc09f9
remove PAPI feature flag (#2601) 2023-12-08 14:55:45 +01:00
mmetc
4acb4f8df3
cwhub: context type (#2631)
* add hub type "context"
* cscli lapi: log.Fatal -> fmt.Errorf; lint
* tests for context.yaml
* load console context from hub
* original & compiled context
* deprecate "cscli lapi context delete"
$ cscli lapi context delete
Command "delete" is deprecated, please manually edit the context file.
* cscli completion: add appsec-rules, appsec-configs, explain, hubtest
2023-12-07 16:20:13 +01:00
Thibault "bui" Koechlin
8cca4346a5
Application Security Engine Support (#2273)
Add a new datasource that:
- Receives HTTP requests from remediation components
- Apply rules on them to determine whether they are malicious or not
- Rules can be evaluated in-band (the remediation component will block the request directly) or out-band (the RC will let the request through, but crowdsec can still process the rule matches with scenarios)

The PR also adds support for 2 new hub items:
- appsec-configs: Configure the Application Security Engine (which rules to load, in which phase)
- appsec-rules: a rule that is added in the Application Security Engine (can use either our own format, or seclang)

---------

Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
Co-authored-by: mmetc <92726601+mmetc@users.noreply.github.com>
Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-12-07 12:21:04 +01:00
mmetc
1ab4487b65
cscli hub list: show only non-empty tables with -o human
* agent config: remove unused LintOnly bool
* Item.IsLocal() -> Item.State.IsLocal(); split method InstallStatus()
* cscli hub list: show only non-empty tables with -o human
2023-12-05 13:38:52 +01:00
mmetc
ffcab0b2bc
Refactor hub management and cscli commands (#2545) 2023-11-24 15:57:32 +01:00
mmetc
5cd4406f5e
typos/grammar (#2561) 2023-11-07 15:07:36 +01:00
mmetc
a6b55f2b5e
cscli config feeature-flags: point user to the right location of feature.yaml (#2539) 2023-10-13 09:52:51 +02:00
mmetc
3b1563a538
Refact cscli hub / pkg/cwhub (part 6) (#2524)
* hub.ConfigDir -> hub.InstallDir; hub.DataDir -> hub.InstallDataDir
* cleanup GetInstalledItemsAsString()
* lint: ReferenceMissingError -> ErrMissingReference
* lint: parent_dir -> parentDir
* link: export Walker type
* lint: return error last
* lint: shadow
* move around and group variable definitions
2023-10-09 21:33:35 +02:00
mmetc
9ae8bd79c5
Refact pkg/csconfig tests (#2526)
* remove unused method
* whitespace, redundant comments
* use test helpers
* move DumpConsoleConfig() from pkg/csconfig to cscli
* package doc header
* var -> const
* rename ./tests -> ./testdata
* shorter tests with more error checks
* lint/formatting
* use helpers; fix tests that didn't actually test
* lint; rename expectedResult -> expected
2023-10-09 11:10:51 +02:00
mmetc
338141f067
Refact cscli hub / pkg/cwhub (part 5) (#2521)
* remove unused yaml tags
* cscli/cwhub: deduplicate, remove dead code
* log.Fatal -> fmt.Errorf
* deflate utils.go by moving functions to respective files
* indexOf() -> slices.Index()
* ItemStatus() + toEmoji() -> Item.status()
* Item.versionStatus()
* move getSHA256() to loader.go
2023-10-06 13:59:51 +02:00
mmetc
bfda483c0a
fix issue #2499 - nil dereference while using capi whitelists (#2501) 2023-10-02 11:42:17 +02:00
mmetc
fd94e2c056
refactor alert/decisions insert/update to avoid database locking in bulk operations (#2446) 2023-09-04 14:21:45 +02:00
mmetc
25868f27de
option db_client.decision_bulk_size (#2440) 2023-08-25 17:05:17 +02:00
mmetc
afeb541eac
apic: minor refactoring (#2415)
* apic: minor refactoring

* Add whitelist length check

If user configures the file but fails to define and actual whitelist we should check length to save allocs

* Init with length from file

* extract loop method from ApplyApicWhitelists

* pass pointer

* extract loop method updateBlocklist

---------

Co-authored-by: Laurence Jones <laurence.jones@live.co.uk>
2023-08-10 13:03:47 +02:00
mmetc
ffadd42779
update dependency on go-cs-lib; drop the pkg/ part (#2393) 2023-07-28 16:35:08 +02:00
mmetc
ae53c0f1cc
fix "crowdsec-cli/require" log verbosity (#2390) 2023-07-28 09:56:20 +02:00
mmetc
5cb7013575
Check cscli preconditions with crowdsec-cli/require package (#2388) 2023-07-27 17:02:20 +02:00
mmetc
9beb5388cb
errors.Wrap -> fmt.Errorf; clean up imports (#2301) 2023-06-23 14:04:58 +02:00
mmetc
3cc6b2c0d0
CI: add tests for metrics configuration (#2251) 2023-06-05 23:17:30 +02:00
mmetc
396dcf8e6e
dependencies: replaced function calls to pkg/types, errors.Wrap (#2235)
we now use a generic pointer function, and slowly remove the deprecated pkg/errors
2023-06-01 16:31:56 +02:00
mmetc
b2d3520519
decouple bouncer dependencies: use go-cs-lib in test code (#2229) 2023-05-25 15:37:44 +02:00
Laurence Jones
0416a41d58
Log info capi whitelists (#2220)
* add infof command if err was nil

* Fix golint

* Make message more readable and log individual stats

* Missed a d

* Remove '

* simplify if/else logic

---------

Co-authored-by: Marco Mariani <marco@crowdsec.net>
2023-05-25 10:28:08 +01:00
mmetc
025f14f879
merge system cert pool with own certs (#2226) 2023-05-25 10:10:58 +02:00
mmetc
534328ca30
decouple bouncer dependencies: use go-cs-lib/pkg/* (#2216)
* decouple bouncer dependencies: use go-cs-lib/pkg/trace
* decouple bouncer dependencies: use go-cs-lib/pkg/version
* decouple bouncer dependencies: use go-cs-lib/pkg/yamlpatch
* decouple bouncer dependencies: use go-cs-lib/pkg/csstring
* unused import
2023-05-23 10:52:47 +02:00
mmetc
e1f5ed41df
Implement "cscli config show-yaml" (#2191) 2023-05-11 21:01:13 +02:00
mmetc
0c5d233563
Minor cleanup and dead code removal (#2166) 2023-04-12 16:57:38 +02:00
mmetc
38ab6be7c2
Allow feature.yml to change available subcommands (#2156) 2023-04-03 10:11:56 +02:00
Thibault "bui" Koechlin
a74e424d53
support ip and cidr based whitelists for capi and 3rd party blocklists (#2132)
* support ip and cidr based whitelists for capi and 3rd party blocklist
2023-03-21 11:50:10 +01:00
blotus
b1f2063a9a
Only support pgx driver for postgresql (#2118) 2023-03-16 11:02:31 +01:00
blotus
16a3be49e2
do not try to load PAPI is url is not set (#2099) 2023-03-06 15:38:58 +01:00
Laurence Jones
8acce4637a
Option to disable remote lapi registration (#2010)
* Allow to disable remote lapi registration

* Extract method and make it extendable as a generic middleware

* Change method name so it make sense to read abort remote if <config>

* golint
2023-02-24 13:44:21 +00:00
Thibault "bui" Koechlin
e927717fa0
Polling API Integration (#1715)
Co-authored-by: alteredCoder <kevin@crowdsec.net>
Co-authored-by: he2ss <hamza.essahely@gmail.com>
Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-31 14:47:44 +01:00
mmetc
e37d09e5b4
use helpers for shorter tests, add a couple of error cases (#2016) 2023-01-26 17:13:31 +01:00
mmetc
47cc60bda9
allow use of literal $ in config.yaml (#2012) 2023-01-23 10:29:29 +01:00
Thibault "bui" Koechlin
4f29ce2ee7
CTI API Helpers in expr (#1851)
* Add CTI API helpers in expr
* Allow profiles to have an `on_error` option to profiles

Co-authored-by: Sebastien Blot <sebastien@crowdsec.net>
2023-01-19 08:45:50 +01:00
Marco Mariani
4f25738d6b wip 2023-01-18 15:15:18 +01:00
Marco Mariani
47dbfa770d configure logging earlier 2023-01-18 15:15:18 +01:00
Marco Mariani
91b0f8fee1 load custom configuration paths when agent is disabled 2023-01-18 15:15:18 +01:00
Marco Mariani
2e91a82aa7 load feature.yaml as soon as possible 2023-01-18 15:15:18 +01:00
mmetc
51800132cd
improve feature flag logging (#1986)
For cscli: it should provide a terse output, not nag users with configuration details. Although it's usually important that cscli and crowdsec have the same enabled features, having it list them every time the command is invoked can be too much.

For crowdsec: when features are set from the environment, it's too early to log where we should. So we can use log.Debug at activation time, and list them again once logging is configured.

 - wrap some functions in csconfig for convenience and DRY
 - for each enabled feature, log.Debug
 - log all enabled features once as Info (crowdsec) or Debug (cscli)
 - file does not exist -> log.Trace
2023-01-13 13:42:42 +01:00
mmetc
cd4dabde0e
silence yaml.local explicitly in cscli, keep in crowdsec/bouncer logs (#1981) 2023-01-11 09:50:46 +01:00
AlteredCoder
185f9ad541
Alert context (#1895)
Co-authored-by: bui <thibault@crowdsec.net>
2023-01-04 16:50:02 +01:00
mmetc
72c1753fb7
fix tls communication with lapi and user/pw auth (#1956)
allow self-signed TLS encryption with user/pw auth

docker:
 - remove defaults for certificate file locations
 - new envvar INSECURE_SKIP_VERIFY
 - register agent before TLS settings (cscli machine add removes them
   from the credentials file)
2022-12-29 22:00:11 +01:00
Laurence Jones
fe23da6e0c
Add postgres socket support, clean some code (#1926) 2022-12-12 16:08:19 +00:00
mmetc
cc228f1868
Typos, grammar (#1905) 2022-12-06 15:55:27 +01:00