You can use what ever you want :) the feed format is really just about the entry encoding, wether you use zip, git, flat files , sqlite or something else is the implementations concern. #go-ssb uses a offset file to store messages, just like with the other format. The #planetary app uses a sqlite db just for some aggregate query/view stuff like reply counts and sorting by timestamp.

If you checkout the go-ssb code and run the tests of the sbot package, it will create a folder testrun with the files of each test (a folder with the name of the test in there etc). This is an easy way to get hold of some fresh feed data.

# get the source
git clone https://github.com/cryptoscope/ssb go-ssb

# open sub-package sbot
cd go-ssb/sbot

# run a simple feed sync test to get the repo data
go test -run FeedsGabbySync

# outputs lots of stuff and should PASS after about 4-5 seconds
...
level=warn t=3.305844514s unit=bob event="idx server exited" idx=contacts error=null
level=warn t=3.305873002s unit=ali event="idx server exited" idx=contacts error=null
level=debug t=3.305888683s unit=ali event="sbot closing" msg="waited for indexes to close"
level=info t=3.315682309s unit=ali event="sbot closing" msg="closers closed"
level=debug t=3.315759236s unit=bob event="network listen loop exited"
level=debug t=3.315768127s unit=bob event="sbot closing" msg="connections closed"
level=debug t=3.315778354s unit=bob event="sbot closing" msg="waited for indexes to close"
level=info t=3.330921373s unit=bob event="sbot closing" msg="closers closed"
PASS
ok      go.cryptoscope.co/ssb/sbot    4.050s


# go into the repo of one of the bots from that testrun
cd testrun/TestFeedsGabbySync/bob && ls
blobs    indexes    log    sublogs

Similar to flume, the log folder is our offset2 "file". Indexes and sublogs are similar to the leveldb kv databases. So to see some actual gabby grove bytes we just iterate through the log file. Here is a dd and xxd/hexdump only way to access the first entry:

# get the first unit64 from the file, the length of the first entry
dd if=log/data bs=8 count=1 | xxd
00000000: 0000 0000 0000 0117                      .......!

# big endian, so it's 279 bytes long (since hexdump is base 16)
dd if=log/data bs=1 skip=8 count=279 | xxd
00000000: 0283 5858 85f6 d904 1a58 2101 6511 af7b  ..XX.....X!.e..{
00000010: 2caf 1283 6422 129e 4388 b929 33b4 3cc9  ,...d"..C..)3.<.
00000020: 6bea 3738 6413 cd97 ec4c 8317 011a 5e03  k.78d....L....^.
00000030: 7ac8 83d9 041a 5821 03d5 b6b1 d5e9 d3bb  z.....X!........
00000040: 3e6d 9296 243a 4122 f9ee ddd9 4ee3 2b52  >m..$:A"....N.+R
00000050: 178e 4f71 4e4c 0d08 0818 7701 5840 f9b4  ..OqNL....w.X@..
00000060: a45c 1ddf 0825 26a0 ef97 4d8f 90d6 61e5  .\...%&...M...a.
00000070: 2da6 f7d8 3c50 7083 04b4 ea7a 7a53 c9d9  -...<Pp....zzS..
00000080: bd6e d78e 7edc 72f3 7c6c 07ca 6e59 46d2  .n..~.r.|l..nYF.
00000090: 390e e6f2 58e8 ba9a 8acd 4d1d c606 5877  9...X.....M...Xw
000000a0: 7b22 7479 7065 223a 2263 6f6e 7461 6374  {"type":"contact
000000b0: 222c 2263 6f6e 7461 6374 223a 2240 5162  ","contact":"@Qb
000000c0: 496f 336b 572b 4e47 5238 2b57 654f 4138  Io3kW+NGR8+WeOA8
000000d0: 3835 7334 6e45 316d 7956 674c 4365 7650  85s4nE1myVgLCevP
000000e0: 6e48 4a37 5352 6538 343d 2e65 6432 3535  nHJ7SRe84=.ed255
000000f0: 3139 222c 2266 6f6c 6c6f 7769 6e67 223a  19","following":
00000100: 7472 7565 2c22 626c 6f63 6b69 6e67 223a  true,"blocking":
00000110: 6661 6c73 657d 0a                        false}.

The first byte here (0x02) denotes the internal multimsg type tag (1 is old/legacy, 2 is gabbygrove) and the rest is valid CBOR. It is the first message the test bot publishes (following the other feed in the old, legacy format). If you throws these hex bytes into cbor.me or the diag tools (ruby..) you can get an annotated version:

83                                      # array(3)
   58 58                                # bytes(88)
      85F6D9041A5821016511AF7B2CAF12836422129E4388B92933B43CC96BEA37386413CD97EC4C8317011A5E037AC883D9041A582103D5B6B1D5E9D3BB3E6D9296243A4122F9EEDDD94EE32B52178E4F714E4C0D0808187701 
   58 40                                # bytes(64)
      F9B4A45C1DDF082526A0EF974D8F90D661E52DA6F7D83C50708304B4EA7A7A53C9D9BD6ED78E7EDC72F37C6C07CA6E5946D2390EE6F258E8BA9A8ACD4D1DC606 
   58 77                                # bytes(119)
      7B2274797065223A22636F6E74616374222C22636F6E74616374223A22405162496F336B572B4E4752382B57654F4138383573346E45316D7956674C436576506E484A3753526538343D2E65643235353139222C22666F6C6C6F77696E67223A747275652C22626C6F636B696E67223A66616C73657D0A

The three arrays form the transfer object and are event, signature and content. The specs should explain how to chop up the rest.

Now let me try to whip up some margaret-find.go on the train to #36c3 :)

Here is what i whipped up: margaret-dump.go

$ go run margaret-dump.go ali/log


Message00: %ULwTkUWOON5exNLOIRrWvgdzr1zp468GO/jwwc8M1xM=.sha256
Author: @QbIo3kW+NGR8+WeOA885s4nE1myVgLCevPnHJ7SRe84=.ed25519
Content: {
    "type": "contact",
    "contact": "@ZRGveyyvEoNkIhKeQ4i5KTO0PMlr6jc4ZBPNl+xMgxc=.ggfeed-v1",
    "following": true,
    "blocking": false
  }
Not a gabby message, skipping...


Message01: %2vSMZwV/t2ZsmSW7RhOlYkcZIadNys0d1Ds7sOHJzuw=.ggmsg-v1
Author: @ZRGveyyvEoNkIhKeQ4i5KTO0PMlr6jc4ZBPNl+xMgxc=.ggfeed-v1
Content: {"type":"contact","contact":"@QbIo3kW+NGR8+WeOA885s4nE1myVgLCevPnHJ7SRe84=.ed25519","following":true,"blocking":false}

evt bytes: 85f6d9041a5821016511af7b2caf12836422129e4388b92933b43cc96bea37386413cd97ec4c8317011a5e037ac883d9041a582103d5b6b1d5e9d3bb3e6d9296243a4122f9eeddd94ee32b52178e4f714e4c0d0808187701


Message02: %Gs0JO2iZb5o85fyY6M7BZhreZh2diCYgVMcMUqzNMrM=.ggmsg-v1
Author: @ZRGveyyvEoNkIhKeQ4i5KTO0PMlr6jc4ZBPNl+xMgxc=.ggfeed-v1
Content: {"test":0}

evt bytes: 85d9041a582102daf48c67057fb7666c9925bb4613a562471921a74dcacd1dd43b3bb0e1c9ceecd9041a5821016511af7b2caf12836422129e4388b92933b43cc96bea37386413cd97ec4c8317021a5e037ac883d9041a582103d0e907167db37c1164d2035c0eaa99dbf63209f3c903dcd96a54900db2b0dc6c0b01



$ go run margaret-dump.go bob/log

Message00: %2vSMZwV/t2ZsmSW7RhOlYkcZIadNys0d1Ds7sOHJzuw=.ggmsg-v1
Author: @ZRGveyyvEoNkIhKeQ4i5KTO0PMlr6jc4ZBPNl+xMgxc=.ggfeed-v1
Content: {"type":"contact","contact":"@QbIo3kW+NGR8+WeOA885s4nE1myVgLCevPnHJ7SRe84=.ed25519","following":true,"blocking":false}

evt bytes: 85f6d9041a5821016511af7b2caf12836422129e4388b92933b43cc96bea37386413cd97ec4c8317011a5e037ac883d9041a582103d5b6b1d5e9d3bb3e6d9296243a4122f9eeddd94ee32b52178e4f714e4c0d0808187701


Message01: %Gs0JO2iZb5o85fyY6M7BZhreZh2diCYgVMcMUqzNMrM=.ggmsg-v1
Author: @ZRGveyyvEoNkIhKeQ4i5KTO0PMlr6jc4ZBPNl+xMgxc=.ggfeed-v1
Content: {"test":0}

evt bytes: 85d9041a582102daf48c67057fb7666c9925bb4613a562471921a74dcacd1dd43b3bb0e1c9ceecd9041a5821016511af7b2caf12836422129e4388b92933b43cc96bea37386413cd97ec4c8317021a5e037ac883d9041a582103d0e907167db37c1164d2035c0eaa99dbf63209f3c903dcd96a54900db2b0dc6c0b01


Message02: %ELXSS7J4KCAvSxbI2tjBnB4/iT5OAVyZwTa9B1DOjrk=.ggmsg-v1
Author: @ZRGveyyvEoNkIhKeQ4i5KTO0PMlr6jc4ZBPNl+xMgxc=.ggfeed-v1
Content: {"test":1}

evt bytes: 85d9041a5821021acd093b68996f9a3ce5fc98e8cec1661ade661d9d88262054c70c52accd32b3d9041a5821016511af7b2caf12836422129e4388b92933b43cc96bea37386413cd97ec4c8317031a5e037ac883d9041a58210376ee56ae9188eecf6269efd1d512d2fb3711524325959737507ea87047355a640b01

again, if you throw the evtBytes into the diag tools you get a nice run down of what is what:

85                                      # array(5)
   D9 041A                              # tag(1050)
      58 21                             # bytes(33)
         021ACD093B68996F9A3CE5FC98E8CEC1661ADE661D9D88262054C70C52ACCD32B3
   D9 041A                              # tag(1050)
      58 21                             # bytes(33)
         016511AF7B2CAF12836422129E4388B92933B43CC96BEA37386413CD97EC4C8317
   03                                   # unsigned(3)
   1A 5E037AC8                          # unsigned(1577286344)
   83                                   # array(3)
      D9 041A                           # tag(1050)
         58 21                          # bytes(33)
            0376EE56AE9188EECF6269EFD1D512D2FB3711524325959737507EA87047355A64 
      0B                                # unsigned(11)
      01                                # unsigned(1)

Here is a diff for turning the above script into a margaret-find.go tool that searches for a message key. Since we don't store the references as strings in base64 we need to compute the ref for each message again...

24,25c24,25
<     if len(os.Args) != 3 {
<         check(fmt.Errorf("usage: margret-find <log dir> <searchkey>"))
---
>     if len(os.Args) != 2 {
>         check(fmt.Errorf("usage: margret-dump.go <log dir>"))
35,39d34
<     // make sure we get a ref we understand
<     // and parse it means we can compare it without encoding it up to b64 again..!
<     searchRef, err := ssb.ParseMessageRef(os.Args[2])
<     check(err)
<
44,47c39
<     src, err := ofstLog.Query(
<         margaret.Reverse(true), // read last entries first
<         margaret.SeqWrap(true), // also get the sequence (so we don't have to keep track of it ourselvs)
<     )
---
>     src, err := ofstLog.Query(margaret.Limit(3))
49a42
>     i := 0
60,67c53
<         // go abstraction magic, now first unbox the sequence-value pair
<         sw, ok := msgv.(margaret.SeqWrapper)
<         if !ok {
<             check(fmt.Errorf("unexpected type: %T", msgv))
<         }
<         msgv = sw.Value() // message inside here
<
<         // now get the message type that's nice to work with
---
>         // go abstraction magic, get a message type that's nice to work with
73,79d58
<         i := sw.Seq()
<
<         if !msg.Key().Equal(*searchRef) {
<             // fmt.Println("not found, skipping",i)
<             continue
<         }
<
81c60,61
<         fmt.Printf("\n\nFound Message%02d: %s\n", i, msg.Key().Ref())
---
>         fmt.Printf("\n\nMessage%02d: %s\n", i, msg.Key().Ref())
>         i++