[Emerging-Sigs] GoNIDS and rule parsing (was:`former_category` metadatas)

Nathan nathan at packetmail.net
Wed Jun 17 04:14:58 HDT 2020


Thanks for your detailed response, I guess I've never run into this
problem statemenet before and because of the engine differences between
Snort and Suricata and many caveats, exceptions, and differences in
the way they operate I tend to hand optimize.

More FOSS is always good though, so nice work :)

Cheers,
Nathan Fowler

On Tue, 16 Jun 2020
19:30:41 -0700 Duane Howard <duane.security at gmail.com> wrote:

> Forking thread with a more appropriate subject line so folks can
> filter as they please.
> 
> Not a stupid question Nathan. The reason I suggest in this context is
> that I've written this library to address rule modification and
> management at scale, which is explicitly what Proofpoint does, and
> the "rule creation engine" they use has been mentioned in a few
> threads I've had. This library is relevant to improving that
> situation.
> 
> More generally, GoNIDS was written as a Suricata specific parser and
> manipulation library for several reasons. I'll list a few here:
> 
>    1. The largest is that manipulation of rules in a programmatic
> fashion using regular expressions in configuration is error prone,
> inflexible and untestable in a sane fashion.
>    2. Generation of rules from data has historically been done in an
> ad-hoc fashion. The library allows you to craft a rule by populating
> a struct in a programmatic, repeatable way. There are a few examples
> linking this library in on github generating rules.
>    3. Programmatic applications of logic. Similar to point 1 we can
> take the differences in the engines[0] and turn them into code,
> allowing someone to:
>       1. Write a rule for one engine, then optimize it for others.
> (Snort -> Suricata 4.0)[1]  
>       2. Optimize across versions of engines. (Suricata 4.0 ->
> Suricata 5.0)[2]
>    4. Utilizing rule metadata in other code. Simple examples involve
>    extracting content matches, and their modifiers and turning them
> into a query that can be applied to a log to look back over time.
> (Imagine running all new rules over your old proxy logs, roughly
> applying new intelligence backwards in time).
>    5. Rule linting/validation without the engine. Validate your rules
>    without running the full blown engine, integrate it into whatever
> IDE plugin you want.
>    6. Consistent feel. Consistent string output guarantees that rules
> look the same, this makes it easier for folks to digest a rule if
> they're not used to interpreting them.
> 
> I hope this sheds some light on why the GoNIDS library exists, and I
> hope that if you have a need to automate processes around IDS rules
> (as I have) it can be helpful to you as well (hence open sourcing
> it). Happy to answer other questions you may have.
> 
> [0]
> https://suricata.readthedocs.io/en/latest/rules/differences-from-snort.html
> [1] https://github.com/google/gonids/blob/master/optimize.go#L73
> [2] https://github.com/google/gonids/blob/master/optimize.go#L154
> 
> On Tue, Jun 16, 2020 at 7:01 PM Nathan <nathan at packetmail.net> wrote:
> 
> > Stupid question here -- why?  If Suricata and Snort parse the rule
> > who cares what Go/Google/Gonads does?
> >
> > 'gonads is a library to parse IDS rules for engines like Snort and
> > Suricata.'
> >
> > Why do I care about this 3rd party implementation for rule parsing?
> > Why should I care about other peoples gonads...er gonids?
> >
> > Cheers and all the best!
> >
> > Nathan Fowler
> >
> >
> >
> >
> > On Tue, 16 Jun 2020 15:29:44 -0700
> > Duane Howard <duane.security at gmail.com> wrote:
> >  
> > > Shameless plug again for GoNIDS
> > > https://github.com/google/gonids
> > >
> > > It'll fix this for you automatically
> > > ```
> > > r := gonids.ParseRule(ruleString)
> > > fmt.Println(r)
> > > ```
> > >
> > > If you find bugs, I'll fix them... =)
> > >
> > > On Tue, Jun 16, 2020 at 3:00 PM Richard Gonzalez
> > > <rgonzalez at proofpoint.com> wrote:
> > >  
> > > > Thanks Duane - this is a formatting issue within our rule
> > > > creation engine and how it deals with out internal and external
> > > > metadata tags.
> > > >
> > > > We're working on making the released rule content a little more
> > > > 'graceful' for these purposes.
> > > >
> > > >
> > > > Thanks,
> > > >
> > > >
> > > > Rich
> > > >
> > > >
> > > > ------------------------------
> > > > *From:* Emerging-sigs
> > > > <emerging-sigs-bounces at lists.emergingthreats.net> on behalf of
> > > > Duane Howard <duane.security at gmail.com> *Sent:* Tuesday, June
> > > > 16, 2020 3:40 PM *To:* emerging-sigs at lists.emergingthreats.net <
> > > > emerging-sigs at lists.emergingthreats.net>
> > > > *Subject:* [Emerging-Sigs] `former_category` metadatas
> > > >
> > > > Stylistic question:
> > > >
> > > > Is it intended that `former_category` metadata tags are
> > > > independent of the other metadata tag in a given rule? Why not
> > > > merge them into a single one?
> > > >
> > > > For example:
> > > >
> > > > alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"ET
> > > > CURRENT_EVENTS SunDown EK RIP Landing M1 B641";
> > > > flow:established,from_server; file_data;
> > > > content:"|4a694270626e525562314e30636968685a4752794b|";
> > > > *metadata: former_category CURRENT_EVENTS;*
> > > > classtype:trojan-activity; sid:2024353; rev:2;
> > > > *metadata:affected_product
> > > > Windows_XP_Vista_7_8_10_Server_32_64_Bit, affected_product
> > > > Web_Browser_Plugins, attack_target Client_Endpoint, deployment
> > > > Perimeter, tag Exploit_Kit_Sundown, signature_severity Major,
> > > > created_at 2017_06_07, malware_family Exploit_Kit, updated_at
> > > > 2017_06_07;*)
> > > >
> > > > It also seems that this particular meta has a space following
> > > > the `metadata` keyword where the later ones do not.
> > > >  
> >
> >  



More information about the Emerging-sigs mailing list