scuttle-shell

name to be bikesheded later

@matt and I are getting excited about this small step:

• scuttle-shell runs in your system tray
• all clients starting connect to scuttle-shell as their backend
• client register the plugins they want (name + version)-spaced, and serves up the goodness

oh yes, please!

Is there a concise explanation of plugins someone could link to? Are they a general part of ssb, or are they an implementation detail of sbot? Do they run in-process of the database or do they communicate via ipc (or even muxrpcs)? Can they be written in any language or only in javascript?

And also: What do they do, why do they exist? They seem to cause most of the pain when trying to run applications without an embedded sbot. Does scuttle-shell fix these problems? Is scuttle-shell language-agnostic or tied to js? If scuttle-shell fixes those problems, why is it tied to a graphical desktop via the system-tray? Or is it just a gui for plugin management?

@aljoscha they are part of the javascript process. They allow people to extend the sbot database without having to debate wether a given thing should be a core feature, most plugins would not have been merged as pull requests because it would make sbot totally unweildly!

I hope one day we have some clever sandboxing thing that handles all the plugins and that we can indeed write plugins in other languages than javascript, but I don't know how that will work yet. (I think wasm will be important here though)

@dominic Web Workers are actually a decent way of achieving sandboxing. I know of a frontend developer doing that for (third-party) plugins in the frontend (as an alternative to iframes), for security purposes.

We could try using that for scuttlebot. There are some Worker polyfills for node: https://github.com/avoidwork/tiny-worker and I already built all the libraries for muxrpc/multiserver over Workers: https://www.npmjs.com/package/multiserver-worker and https://github.com/staltz/pull-worker

@Dominic Does that mean that none of the clients relying on plugins (i.e. all of them) will work with a non-js ssb-server implementation?

@aljoscha it means that the alternative implementation needs to implement that plugin too... if we made the plugin speak over muxrpc then we could have multilanguage plugins, there are also great security reasons for doing this!

A plugin could have a permissions list of what it is or isn't able to access (and it's easier to audit the permissions list than the code - my wifi manager shouldn't need to see contacts list, file system, or connect to internet, etc). Android theoretically has this, but it sucks because it's basically an ultimatum, either you agree to the permissions or the app doesn't install. And, connecting to the internet isn't a special permission! it ought to be, but then ads in every app wouldn't be possible (they shouldn't be)

Nowadays Android (Oreo, I guess) permissions are fine-grained and you can pick which ones, also during usage time, not on install only.

oh heads up this is meant as a step in the direction that's better than where we are now! it's gonna be wrong and not do some things and limited, and we'll learn heaps.
I wanted to write this down as a mental note from the conversation with @matt because we were both feeling pretty good about it and thought it would help the app ecosystem a lot. We talked about a bunch of other out there ideas but the point is to get the ball rolling, not boil the ocean :)

I have parts of this already working due to my work on #patchfox. I have decoupled my front-end webextension from my back-end nodejs sbot. I can run them independently.

The current NodeJS back-end runs as a tray app alreay as can be seen below:

It gives nice notifications like:

So, what I am doing now is considering a new ssb-ws-rpc plugin with a simple JSON based interface that would allow apps to invoke sbot features from this tray app. Thats how I am building the Firefox add-on for the February Grant. Work is being placed on this tray app as it benefits everyone.

The server discovery plugin I mentioned in this thread also ties into this.

cc @mix and @Matt McKegg

A more interactive demo of the current workflow

It shows running the sbot tray app and a webextension requesting permissions to connect.

Anyone has feedback on it?

how is the scuttlebutt in tray managed / started. Is it an electron thing? It looks like you're starting it in powershell perhaps ?

Would love to see the repo for what you've built, where's your most current work live @SoapDog ?

@mix,

It is not electron, it is a pure NodeJS app that uses some NPM modules that are binary apps for handling tray and notification, this way they are smaller than a full-blown electron app.

I am using powershell just as a terminal, it is just an normal nodejs app, you can start it from bash or whatever way you want. I am still testing but I think that Firefox can start that app as well if it is not running (I haven't tried making Firefox start any app that have a graphical UI yet).

All the code is at https://github.com/soapdog/patchfox, the tray app is the host_app directory. The server.js file contains everything.

That repo also contains the webextension which is in transition right now.

awesome!

awesome ... also awesome name !

Would this enable running multiple clients at once (such as ticktack, patchwork, patchbay, chess)? :o

Exciting times

Scuttle-Shell

Following the motto of Do instead of Ask and a ton of advise from many of you, I went ahead and published a little Scuttle-Shell application as an NPM module. This enables you to run sbot as a system tray app (or the equivalent in your OS of choice).

In the future apps can just check if sbot is running and if it is not, it is very easy to detect if scuttleshell (the shell cmd has no hifen because reasons) is installed and launch it. Right now, the only app that does it is patchfox.

I have many features planned for this app but this is a small and easy starting point. All this code has been extracted from the patchfox repo and refactored a bit, so I do count this as part of the work built from my #ssb-grants even though it is no longer tied to #patchfox.

The installation procedure is now much easier, installing the module should handle everything. You can install it globally which will provide a scuttleshell command to launch the app, or, clone the repo and run node server.js to start it manually (Patchfox can find and start the app on its own, no need to start it on your own).

All the new source code is on the Scuttle-Shell Github Repo and I will soon patch the patchfox repo to use this repo instead of its own for the host app.

cc #ssb-show-and-tell

Scuttle-Shell Tray app running on windows:

Scuttle-Shell Menubar app running on macOS:

PS: Need to find a way to remove the app title from that menu... these are early days

Scuttle-Shell is super important! Thanks @SoapDog for doing it, and I'm sure we'll keep improving it little by little. SSB needs an "operating system" with plugins (built in any language) and support for multiple UI apps, so this is the right way to go. I'll try installing it on Ubuntu and see how it goes

I'd be interested to collaborate and hack with you on this a little bit @SoapDog , if that interests you.

:) I have a lot of JS experience, but not a lot of SSB experience.

Could I pick up that app title aspect?

I've pushed a copy of this up to git-ssb : ssb://%AVhVlwiF+1XgnJMWJYxGbUP9mG87kzoAuwfmho/stYc=.sha256

I'm working on getting it running on linux. I'l be pushing a branch up there when I do!

hey @Joey STANFORD can you post errors on the repo issues please ? It's easie rto discuss there : https://github.com/soapdog/scuttle-shell/issues

Also this is a super early release and you've just dumped a NOPE on a little sappling that's trying to get started, without saying anything about what you apparently enjoy / didn't enjoy, expressing anyhting other than CODE OUTPUT. I'm exploring what it's like to support the humans first, and the code secondarily. Because there is no code if humans decide it's not worth their time, or it's too hard to share. And I want everyone to be able to share, not sure people who are battle-hardened and willing to defend their opinions aggressively (which is one approach we see in FOSS).

Full-disclosure - I am also feeling responsible and sensitive because I encourages SoapDog to post this early, so that we can start hacking on it. And i'm excited to see that at least you me and staltz have all tried installing it, which is an awesome form of support. I'm also working on getting a linux branch that works well for me up on git-ssb. I think this is exciting and an awesome way to grow this thing together, and I don't want SoapDog to regret posting this when he did because. This might seem like an oursized response, but this I feel this is a really important topic because the effect is highly cumulative. Like if 2 more people post Broken, ERROR, CODEDUMP how does that feel, how about when there have already been 3 other post like this on other code you've tried sharing in another repo? Maybe I should just get a paying job or keep my work to myself?

p.s. I'm posting this from scuttle-shell.
If you're on linux, my work so far is on git-ssb ssb://%AVhVlwiF+1XgnJMWJYxGbUP9mG87kzoAuwfmho/stYc=.sha256
on a branch called linux :)

the tray icon works in linux btw
I'm super excited to have hermes hanging out with me.

Yes! I managed to get mix's linux branch working here too (Ubuntu Gnome, showed in the top bar), but I had to edit the scuttleshell.json path field to be my absolute path. I think we could generate that file on the fly instead of using a static one.

@mix pushed your linux branch to the scuttle-shell repo on Github, thanks a lot for the fixes. I will try them tomorrow on windows and mac and if possible merge them back into master.

@Connor awesome!!! I am interested in collaboration, of course!

Don't know what is your OS of choice but if you're on a mac and could help debug that, it will be awesome.

@Joey STANFORD I will double check on the mac tomorrow.

@mix don't worry. I am quite glad that you helped push me towards releasing an early version. I how receptive the community has been about this little tool and think that together we can grow this to be pretty useful.

@mix I am experiencing an odd thing which you might be able to help. I decided to pull from your repo again before doing a PR to my repo since you might have changed something yesterday. When I try to pull, I am receiving this error:

PS C:\Users\soapdog\prog\ssbc\scuttle-shell> git pull
abrt { Error: EPIPE: broken pipe, write
    at Socket._write (internal/net.js:23:7)
    at doWrite (_stream_writable.js:410:12)
    at writeOrBuffer (_stream_writable.js:396:5)
    at Socket.Writable.write (_stream_writable.js:294:11)
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\stream-to-pull-stream\index.js:76:18
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-git-remote-helper\index.js:469:11
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-cat\index.js:35:13
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-git-remote-helper\lib\pkt-line.js:31:9
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-cat\index.js:35:13
    at next (C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-git-remote-helper\index.js:132:11) errno: 'EPIPE', code: 'EPIPE', syscall: 'write' }
C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\git-remote-ssb\git-remote-ssb:58
        if (err) throw err
                 ^

Error: EPIPE: broken pipe, write
    at Socket._write (internal/net.js:23:7)
    at doWrite (_stream_writable.js:410:12)
    at writeOrBuffer (_stream_writable.js:396:5)
    at Socket.Writable.write (_stream_writable.js:294:11)
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\stream-to-pull-stream\index.js:76:18
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-git-remote-helper\index.js:469:11
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-cat\index.js:35:13
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-git-remote-helper\lib\pkt-line.js:31:9
    at C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-cat\index.js:35:13
    at next (C:\Users\soapdog\AppData\Roaming\npm\node_modules\git-ssb\node_modules\pull-git-remote-helper\index.js:132:11)
Already up to date.
PS C:\Users\soapdog\prog\ssbc\scuttle-shell>

Which is odd... sbot is running but I can't pull.

@SoapDog I wonder if it's because I force pushed something D:
Know that the branch is up on github too : git@github.com:mixmix/scuttle-shell.git

I'd be keen to have it in github/ssbc so that I could push to branches if you're open to it.

This is really cool. I want to try it out but I'm stuck. Can anyone help?

@Joey STANFORD this is awesome to hear. It was totally unclear from what you posted what your intention would be. Given I don't know you that well yet, pausing to add those 2 sentences would make a world of difference. It makes you a relateable human and helps me get to know you so much more than some error-reporting

Scuttle-Shell v0.0.3

I've just pushed a new version to NPM, it incorporates the Linux changes by @mix along with some other improvements he made. I've also made some more features, the most important one is that there is a new API for it so that you can require('scuttle-shell') and start or stop the little app from your own app.

let scuttleshell = require("scuttle-shell")

console.log("Starting sbot, quitting after 30 seconds")
scuttleshell.start()

setTimeout(scuttleshell.stop, 30000)

I am still patching #patchfox to work with it. There has been so many changes in how the native host app works that Patchfox has a lot of catching up to do. The booting part of the add-on is so confusing right now that it is hard to keep all the process in your head. I made it to flexible, I am simplifying it and I hope it will make a lot easier for people to use it.

Yes I'm on a mac. Awesome. I will check out the latest version to see where it's at now

I'd like to go ALL IN on scuttle-shell.

Check out this PR :https://github.com/ssbc/patchbay/pull/204

it removes all sbot responsibility from patchbay.
When Patchbay opens, it checks if there's a shell running, and if there is uses it. If not, it spawns a detached scuttle-shell... which keeps running.

I think this will force some really interesting further iterations. Would love input from others before merging this
cc @arj @happy

that link again : https://github.com/ssbc/patchbay/pull/204
branch full-scutle-shell on ssb + github (yes there's a typo in the branch name)

@mix sounds like a planarooni :D

Thanks @SoapDog !

@mix I checked out out the branch. I can still run patchbay with my own sbot so I'm happy :) Actually I'm more happy than before, because npm install is faster :-)