[Emerging-Sigs] GoNIDS and rule parsing (was:`former_category` metadatas)

Duane Howard duane.security at gmail.com
Tue Jun 16 17:30:41 HDT 2020


Forking thread with a more appropriate subject line so folks can filter as
they please.

Not a stupid question Nathan. The reason I suggest in this context is that
I've written this library to address rule modification and management at
scale, which is explicitly what Proofpoint does, and the "rule creation
engine" they use has been mentioned in a few threads I've had. This library
is relevant to improving that situation.

More generally, GoNIDS was written as a Suricata specific parser and
manipulation library for several reasons. I'll list a few here:

   1. The largest is that manipulation of rules in a programmatic fashion
   using regular expressions in configuration is error prone, inflexible and
   untestable in a sane fashion.
   2. Generation of rules from data has historically been done in an ad-hoc
   fashion. The library allows you to craft a rule by populating a struct in a
   programmatic, repeatable way. There are a few examples linking this library
   in on github generating rules.
   3. Programmatic applications of logic. Similar to point 1 we can take
   the differences in the engines[0] and turn them into code, allowing someone
   to:
      1. Write a rule for one engine, then optimize it for others. (Snort
      -> Suricata 4.0)[1]
      2. Optimize across versions of engines. (Suricata 4.0 -> Suricata
      5.0)[2]
   4. Utilizing rule metadata in other code. Simple examples involve
   extracting content matches, and their modifiers and turning them into a
   query that can be applied to a log to look back over time. (Imagine running
   all new rules over your old proxy logs, roughly applying new intelligence
   backwards in time).
   5. Rule linting/validation without the engine. Validate your rules
   without running the full blown engine, integrate it into whatever IDE
   plugin you want.
   6. Consistent feel. Consistent string output guarantees that rules look
   the same, this makes it easier for folks to digest a rule if they're not
   used to interpreting them.

I hope this sheds some light on why the GoNIDS library exists, and I hope
that if you have a need to automate processes around IDS rules (as I have)
it can be helpful to you as well (hence open sourcing it). Happy to answer
other questions you may have.

[0]
https://suricata.readthedocs.io/en/latest/rules/differences-from-snort.html
[1] https://github.com/google/gonids/blob/master/optimize.go#L73
[2] https://github.com/google/gonids/blob/master/optimize.go#L154

On Tue, Jun 16, 2020 at 7:01 PM Nathan <nathan at packetmail.net> wrote:

> Stupid question here -- why?  If Suricata and Snort parse the rule who
> cares what Go/Google/Gonads does?
>
> 'gonads is a library to parse IDS rules for engines like Snort and
> Suricata.'
>
> Why do I care about this 3rd party implementation for rule parsing?
> Why should I care about other peoples gonads...er gonids?
>
> Cheers and all the best!
>
> Nathan Fowler
>
>
>
>
> On Tue, 16 Jun 2020 15:29:44 -0700
> Duane Howard <duane.security at gmail.com> wrote:
>
> > Shameless plug again for GoNIDS
> > https://github.com/google/gonids
> >
> > It'll fix this for you automatically
> > ```
> > r := gonids.ParseRule(ruleString)
> > fmt.Println(r)
> > ```
> >
> > If you find bugs, I'll fix them... =)
> >
> > On Tue, Jun 16, 2020 at 3:00 PM Richard Gonzalez
> > <rgonzalez at proofpoint.com> wrote:
> >
> > > Thanks Duane - this is a formatting issue within our rule creation
> > > engine and how it deals with out internal and external metadata
> > > tags.
> > >
> > > We're working on making the released rule content a little more
> > > 'graceful' for these purposes.
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Rich
> > >
> > >
> > > ------------------------------
> > > *From:* Emerging-sigs
> > > <emerging-sigs-bounces at lists.emergingthreats.net> on behalf of
> > > Duane Howard <duane.security at gmail.com> *Sent:* Tuesday, June 16,
> > > 2020 3:40 PM *To:* emerging-sigs at lists.emergingthreats.net <
> > > emerging-sigs at lists.emergingthreats.net>
> > > *Subject:* [Emerging-Sigs] `former_category` metadatas
> > >
> > > Stylistic question:
> > >
> > > Is it intended that `former_category` metadata tags are independent
> > > of the other metadata tag in a given rule? Why not merge them into
> > > a single one?
> > >
> > > For example:
> > >
> > > alert http $EXTERNAL_NET any -> $HOME_NET any (msg:"ET
> > > CURRENT_EVENTS SunDown EK RIP Landing M1 B641";
> > > flow:established,from_server; file_data;
> > > content:"|4a694270626e525562314e30636968685a4752794b|"; *metadata:
> > > former_category CURRENT_EVENTS;* classtype:trojan-activity;
> > > sid:2024353; rev:2; *metadata:affected_product
> > > Windows_XP_Vista_7_8_10_Server_32_64_Bit, affected_product
> > > Web_Browser_Plugins, attack_target Client_Endpoint, deployment
> > > Perimeter, tag Exploit_Kit_Sundown, signature_severity Major,
> > > created_at 2017_06_07, malware_family Exploit_Kit, updated_at
> > > 2017_06_07;*)
> > >
> > > It also seems that this particular meta has a space following the
> > > `metadata` keyword where the later ones do not.
> > >
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.emergingthreats.net/pipermail/emerging-sigs/attachments/20200616/a3c91e2f/attachment.html>


More information about the Emerging-sigs mailing list