Libre Routers ♥️ 📡
Been a long while since I posted an update about the project.
A while ago I asked for helping getting some LibreRouters into Brazil, and thanks to @SAn, @Cusco and friends of neighbors from Moinho we finally got the first batch of four routers, without any costs, from Buenos Aires, to Porto Alegre and finally arriving at Moinho. Brazilian customs would have probably charged an immense amount of $$.
Three of them have been installed in Moinho, making the mesh infrastructure way more resilient. First and most important was the installation in Aridi's house, which is the where the internet gateway is (pictured above). Another was installed in another main street of Moinho, where a total of 3 routers have failed to maintain a decent connection, so we're hoping the LibreRouter will handle the job. The last was placed close to Aridi, to maintain a mesh at 2.4Ghz, as we haven't gotten the wires and antennas for it. When they arrive we'll find a better place to make use of it's full potential.
The last LibreRouter has come with me to Mato Grosso, where after the pandemic is over, I'm planning on visiting indigenous communities in the region and presenting them the #librerouter, #huiom, #manyverse, #mapeo and #ahau. I believe it will be of great value to them.
Community mapping 👩👩👧👦 🏡
The original vision for the project was to use the WiFi network open around the village so that visitors could see other points of interest of the village besides the waterfall, thus benefiting more people with the existing tourism influx.
Finally took some time to build the map, using @okdistribute's tile-downloader which @Kira helped me to figure out the details, to have offline maps.
I configured #pirania to open up the portal with the map as the captive-portal. There's an "Internet" tab where members of the network can authenticate. There's also a "Local applications" tab where in the future I plan to add some applications, many that use #dat, #ssb and others that @Nico Pace has been curating, mainly that will work on the mesh independent from internet connections.
Community takeover 👬 👭 ✊
Because of this project, I became known as "the internet guy", which is quite the irony, as I'm probably the one in the village who dislikes the internet the most (due to the colonizing effects of the corporate-web). It's been good to be able to help my neighbors with their connectivity problems, but more and more it started becoming a bit of a burden... doing something I don't truly believe in, and being held responsible for internet problems.
So me leaving now for a many months is an opportunity for the other collaborators from the community to maintain the network without me. Apparently it's becoming a bit of a mess as it coincided with the quarantine, which from I've been hearing Moinho is taking seriously, locking down the entrance to the village. But I'm confident they'll unite more, and become more self-reliant, which will be an amazing lesson for them and myself, who am personally interested in how to do #communitynetworks without depending on the figure of a techie person in the community.
Looking forward for how they'll collaborate with each other to maintain their network.
☔ 💧 ⚡ The Rain ☔ 💦 ⚡ 💦
Remember last year's rain? It's that time of year again... and again the first weeks were quite challenging to maintain the community network infrastructure.
While I was chatting with @mixmix on a #ahau meeting a tremendous lightning storm started. It burned many appliances around the village, I'm off-grid, but the infrastructure suffered.
The Libre Router is plugged into a surge protector, but for some reason it's charger burned, at least it was a cheap one, and it can handle an array of voltages, so getting another charger was easy.
We lost a TP Link 3500 router and charger, and something's weird with the network cables that were connected to it, still trying to debug that.
Two of our brand new CPE 510's started acting weird, and it seems they lost networking with Ethernet, which is quite crucial as it only has one 5Ghz radio, for meshing, and is usually connected thru cable to some low-end router. Also their POE chargers burned.
Not related to the lightning strikes, but also lost two CPE 510's version 1, which was acquired by the neighbors to connect a new part of the village... can't
ssh into them anymore after installing LibreMesh firmware. There's probably a fix to them though.
Kinda funny, in one post I got a bunch of equipment, and the next a lose a bunch. That's life.
Spent last week working on this side-project on my spare time. lime-updater is a CLI tool built in NodeJS with oclif, easy install with
npm i -g lime-updater.
Last year I wrote this post while still in Quitana learning the basics. Me, @markitos and @nicoechaniz spent from 1am until 6am updating their huge mesh. Libre Mesh development is an ongoing process, so it shouldn't have to be that hard to update the firmware in the mesh. So I gave myself a week to give a try at solving this problem.
It starts by checking the latest LibreMesh revision, which I found out to be based on the latest lime-packages commit. It ssh's into each node and gets some basic information about them, such as how many hops it is from the node you're running the program from, the ip, and model of the node and it's Libre Mesh revision.
Surprised me that starting ssh sessions in parallel didn't work, had to do some hacky
for..of loops to get synchronous ssh one node at a time.
It then backs up the configs to your computer, selects the right firmware and upload's it to the node. After all that is done it starts updating from the most distant nodes.
This is the most tricky part... after installing the new version it ssh's into each node,
scp's the backup
tar.gz file and restores the old settings.
I was able to update my whole network using this software, but not in a single run as it's intended to work. Found out that there are many edge cases to this whole process depending how the mesh has been configured. Different channels, and virtual interfaces that not always enables the mesh to recover itself after installing the new firmware with no configuration.
But it was a great experiment, definitely learned a lot, and got some great feedback from Marcos and San. If it proves to be useful it'll start being polished thru the magic of #open-source collaboration. It also gave me many ideas for visualization tools for mesh, which are very useful.
Also started collaborating with @brunovianna on a project to run this on mobile and use some kind of gossiping protocol to share firmware among devices, for an offline-first approach.
Let's start from where I last stopped. I had just put up the #librerouter, but it looked like one of the 5Ghz radios or antennas wasn't working properly. I was about to go to coolab camp at @Hiure's community. @SAn would be there, and he's the main engineer behind the hardware as well as the software. Having him close is like having a human LibreMesh/Libre Router documentation with the best query mechanism possible. He's also a really really nice guy. He helped me go over the hardware, and showed me how he tests them.
Basically we ran
iw dev with each of the 5Ghz radios that outputs something like this for the Libre Router:
phy#2 Interface wlan2-mesh ifindex 38 wdev 0x200000005 addr a8:40:41:1c:84:2b type mesh point channel 48 (5240 MHz), width: 40 MHz, center1: 5230 MHz txpower 26.00 dBm phy#1 Interface wlan1-mesh ifindex 42 wdev 0x100000006 addr a8:40:41:1c:84:1d type mesh point channel 157 (5785 MHz), width: 40 MHz, center1: 5795 MHz txpower 27.00 dBm phy#0 Interface wlan0-apname ifindex 27 wdev 0x4 addr ae:40:41:1c:85:28 ssid moinho.digital/morro type AP channel 11 (2462 MHz), width: 20 MHz, center1: 2462 MHz txpower 26.00 dBm Interface wlan0-ap ifindex 22 wdev 0x3 addr aa:40:41:1c:85:28 ssid moinho.digital type AP channel 11 (2462 MHz), width: 20 MHz, center1: 2462 MHz txpower 26.00 dBm Interface wlan0-mesh ifindex 21 wdev 0x2 addr a8:40:41:1c:85:28 type mesh point channel 11 (2462 MHz), width: 20 MHz, center1: 2462 MHz txpower 26.00 dBm
The most important bit of that is the
txpower to check if it's transmitting alright. The Libre Router has 3
phy interfaces, one for each radio, and different virtual interfaces for meshing and access points. Ideally we shouldn't have a radio meshing and being an access point at the same time, but at the hill it's basically not used at all as an AP, and I need it to mesh on 2.4Ghz, as there's a specific house that can only connect to it thru 2.4, and it does help bridge nodes that don't always have a stable 5Ghz mesh connection.
Then we connected each of the antennas and meshed with another device and checked for differences in signal strength with
iw dev wlan-iterface station dump.
We concluded every thing was fine. It was probably me not knowing how to deal very well with different channels. I brought it back to Moinho and it's been running very well since at the top of the hill, connecting now 4 parts of the village to the mesh.
Also added an idea I had discussed with @Nico Pace about adding an extra step with information and a timer for the user to get some intro into the community network information before navigating.
Next step is implementing the Pirania admin interface into Lime App. @markitos has been busy making the code cleaner and easier, thank you for that 🙏
With the last setup, using a wdr-4300 and a parabolic plate for each of it's antennas signal strength for every connection showed good measurements. But slowly we began to suffer connections problems. On this year's Latin American Community Networks Summit @ctrlTroian explained to me how a device with a single 5Ghz radio couldn't handle such task. I then realized the importance of MIMO that @markitos had explained to me long ago but I had forgotten.
Libre Router to the rescue
On the Summit #altermundi handed me the first of six Libre Routers that Moinho will get to test on and report. As soon as I got back home, me and Aridi went up the hill to setup the #librerouter and test how it would compare with the tri-plate setup.
The Libre Router comes with two 5Ghz radios and one 2.4Ghz radio. It ships with two external MIMO sector antennas for the 5Ghz radios and an internal sector antenna for the 2.4Ghz radio. All 4 pigtail connectors come with it, and they're a bit different from the ones we use with the TP-Links, so both me and Aridi thought they were all broken at our first try to hook them up.
The casing is weather-proof, and made from re-purposed casing from other commercial hardware, so it's not as good quality as an usual commercial weather-proof router such as a CPE 510, but good enough to keep it safe and makes it cheaper.
We can power it either over POE or regular power supply, accepting 12-36V. I used the 4300's power supply, and it worked perfectly.
Check the official website for full specs.
I wasn't used to dealing with multiples channels, as I've always had a single radio on each node. So it took me a while to get into the mindset of operating on different channels.
From our initial tests on the top of the hill it seemed one of the 5Ghz radios wasn't working properly. So we took it back down to analyze, and left a single antenna connected.
Amazingly that single sector antenna with it's 23dbi gain was able to connect three different parts of Moinho in a much more stable way then the 4300 and it's plates.
A friend donated four CPE 510 v3, that had just recently gained Libre Mesh support on my request and thanks to @SAn, @moacir and @Hiure. They each have a 5Ghz MIMO radio and weather-proof casing.
Along the year we've gotten many donations from collaborators from the community and also from external friends such as @benhylau, who donated a bunch of cool toys for us after #dweb-camp.
Here's a list of what we have available to play with, besides the nodes we already have in place:
- 4 tp-link CPE 510 (Libre Mesh compatible)
- 1 Ubiquiti Airgrid M5 (not compatible with Libre Mesh)
Dual-band Libremesh compatible 2.4/5Ghz
- 1 tp-link wdr 3600
- 1 tp-link wdr 4300
- 2 water proof enclosers
Acess Point 2.4Ghz not Libremesh compatible
- 2 tp-link 840n
- 1 tp-link 841nd
- 1 generic router
- 1 Rock64 4Gb ram
- 1 Raspberry Pi 3+
- 1 Raspberry Pi Zero
- 1 Espressobin v5
- 1 linksys 8 port + 2 gigabit port switch
- 1 tp-link wn722n usb wifi
- 1 cable crimper
- about 100m of networking cable
- 1 class 10 16Gb micro sd
- 1 class 10 32Gb micro sd
- 2 class 10 64Gb micro sd
- 5 100mb usb drives
- 1 Terabyte usb hard drive
- 2 500Gb usb hard drive
When we began this journey I had a lot of free time, no devices to play with and almost no knowledge. It seems tables have turned. We now have a bunch of toys and experience, but not enough time to play with it all!
Since the last update we've been enjoying a 10Mbs radio connection, but as the map indicated, most of our links were still very poorly made.
Ethernet cable links
Alte Mundi guys had thought me that cable links are the best option if distance is under 107 meters.
Our friends @Luana Marcelle and Felix had bought a 300 meter reel, and we've been holding out on using it, mostly because I suck at crimping the cables. But as Thiago moved back to the village a few months ago, and he's amazing at crimping, we started thinking on how to use the cables to fix some of our links and expand the network.
We moved dino node to a taller house next to his, belonging to his brother Edi, where we have a great line of sight to the hill tower. Within a couple of days we rolled out 130 meters of cable, connecting the newly moved node dinoeedi, passing thru the houses of two new contributors to the network using normal, non-flashed routers, and finally connecting to jesse node, who previously had no decent route to a point-to-point connection.
I plugged the Ethernet cable on jesse node's wan port, and it took me some time to realize how that completely messed up the network, as the node starting broadcasting it was an exit to the internet. Every now and then I had to keep restarting the nodes so that they would eventually find the right exit thru delvaearidi. Bothered our weekly #ahau meeting a bit. But was an easy fix.
Tuning 5Ghz links
The next challenge was adding an extra DIY dish to amplify the signal from the hill to the dinoedi node, now responsible for providing it's whole street with a route to the gateway. Me and Dura went up to the hill, and tried adding an extra dish to the pole we already had in place, but it really didn't fit. Without an adequate plan we stacked them one on top of the other, resulting in lots of interference on one another. Links which had been previously tuned already were messed up again. Our Scuttle-camp 2 meeting suffered a bit from this mistake, @mixmix was very patient in letting me know when I got cut off the call, which was very frequently.
Today me, Aridi and Thiago took the time to go up the hill with a plan and the will to get everything fixed. Using an arm from a broken wheelbarrow that I had laying around, we were able to fix it to the tower and finally fit all the 3 dishes without interference. Thiago had the brilliant idea to disconnect all the antennas, and tune each one at a time, since there's no way to get readings for each individual antenna. With that we found out one of the antennas was damaged. Another easy fix, and this is how our hill node looks like now:
Pretty #solarpunk huh? 😆
It's a single tl-wdr4300 router, with each of it's antennas pointing to a specific node in each of the three parts of the village that we have our collaborators so far.
With these updates, this is how our network looks like now:
Mostly all green 🎉 📡 🎉
The next big challenge we're facing is with Pirania, our captive-portal solution, which is still a headache for most users. Based on the experiences we've had with it for the past few months I'm slowly working on bettering the whole system, and incorporating it's UI into Lime App, with some network governance indicators to give transparency to the network.
Next week the second edition of the Latin American #communitynetworks Summit is happening in Colombia. Time goes by so fast, and I'm pretty excited to meetup with some amazing people from movement again, as the first edition was incredible.
Hope to meet you once again soon @Nico Pace!
Please do @jacob :)
@Luandro Mind if I use the first image in a talk? :)
Current status of the mesh right before the workshop. The mesh has been really unstable, so I'll go to Aridi's house to be directly connected to the gateway. @bobhaugen helped do some quick tests, and we think we can make this work.
Has been an intense week with the network, trying to get it to stabilize
Even with the concrete pole, we noticed that the link between Aridi, who's the gateway to the internet, and the hill was unstable. We we put a thicker and much shorter pole at the highest point of Aridi's house, which is what we should have done from the start. With that the connection finally stabilized despite the wind :wind_blowing_face:
The next problem we found was that the hill node was being turned off every now and then, so the problem wasn't one of signal quality alone. Went up the hill to find that the router was turning the wifi off by it self. LibreMesh is going thru major changes in architecture right now, substituting bmx6 with babel, and using shared-state, a sort of gossiping protocols for mesh nodes, so I though it could be a software problem, although none of the other nodes running the same version had this problem. Since we had a spare router, I decided that would be best to change the router itself, so me and Dura went up there to change the a WDR3500 for a WDR4300 :satellite_antenna:
Final challenge had to do with our energy. I had short-circuited the long wire from the house of Felix, a good friend and collaborator to the network, so we had to ask an untrustworthy old guy, who lives right next to our hill node, to use his electricity. The network had been going off everyday, so we found out he was leaving the house and turning the energy off. So Dura, another collaborator to the project, and Aridi went up there and put and isolator making our energy independent from his. So far so good. We've been thinking on a solar setup, but it's still very expensive here in Brasil, so Dura suggested we put a car battery and make it charge with grid-electricity. That way even if electricity goes down for a few hours, the node won't get turned off. Is there a controller for that?
The network has finally stabilized, so I can to back to testing Pirania once again, since I had to turn if off because of a few bugs. With a more mature shared-state hopefully we can get our captive-portal and distributed voucher system to work!
The concrete tower has been erected!
Had a problem with the node in the tower for the first two days, but everything was back to normal after a firmware re-install this morning. It's really windy now, and at seems the connections has stabilized, but I noticed that Aridi's node is unstable due to the wind. So that's one of the next challenges to tackle, but this operation has been an overall success and milestone for our project.
The connections still aren't the best, relying mostly on the 2.4Ghz radio for two important links, which is not ideal, and is slowing the bandwidth down. Ping on the gateway is at around
30ms while in the tower node it's at ~
350ms, varying a lot.
But as long as the internet is stable enough, people will be happy. I'll update all the nodes with some latest updates to
shared-state, and try our captive-portal voucher system once again. If the bugs start happening I'll have to take down the vouchers and put passwords on the routers, until we solve the issues.
We also had an interesting community meeting about our watershed. Such an assembly hasn't happened since the first year I was here, over 5 years ago, when the state wanted to build a sewage system and throw the waste at our river. We succeeded in stopping the project, but the meeting was really a shame, since the topic was forgotten and small personal issues took over the discussions.
I feel a revival in community union is happening, and information was talked about as key element. Although most are aware of the community-network project, they still haven't made the connection to how it could help with that. Let's give time some time.
Should actually be
Time flies too fast for me to keep up with it.
A few weeks ago we had the first gathering in the neighborhood playground. Was really small, with only 3 people representing each of their nodes, a representative of the Quilombola Association, which I invited, me and @Isabela, and one extra women which was interested in the idea.
The community network's governance is still very tied to me and Aridi, so the intention was to let everyone know that they can get involved as much as they would like, and that all decisions should be taken in meetings, that we'll try to have regularly.
Another topic I brought was that we were invited to collaborate on a project to seek funding with the Mocambos Network, whom @befree is a core contributor. While he was here for a couple of days visiting we drew a plan on how to connect Moinho and Vão do Moleque, a very very isolated Quilombola community that they've connected to the internet thru a series os epical links. Everyone here in Moinho was excited about the idea of strengthening their bonds with the other community thru digital communication. Some of the possibilities that came up during the discussion was on trading goods, since they also produce many organic goods; rescuing ancestrality as a few families here are decedents of these other more isolated communities; and building an Quilombola network infrastructure with multiple gateways, making for a more resilient design.
I'm personally completely disconnected from outside news, most specially national government dramas. I believe in community sovereignty, so local political dramas are what matter to me. Lately the Whatsapp neighborhood group, which was usually for people trying to get rides and helping each other with small favors, has been the stage for major political discussions regarding the village.
An USA expat whose been living here for many years, has suddenly decided to take action on his own for things which should be part of the commons, such as our watershed and security of the village. I personally know very little about him, as most of his relations are only with his employees. The only time I met him he tried to get me to work for him. That kind of people scares the shit outta me, wanting to control everyone with the idea of "helping" thru capital flows. One of his employees is known to be the mayor bootlicker, so he has direct contact with the local government.
Long story short, the whole thing is sort of a mess, as only outsiders are part of the discussions, which happens thru writing, and the natives won't take the time to read the long texts; and small favors are lost amid the long discussion streams. The good thing is that we started having local assemblies, which are still kinda segregated amongst small groups, but are slowly uniting into a try community assembly.
It's the perfect opportunity for presenting an alternative digital cipherspace for the discussions to happen. If I just changed the
caps in both Patchwork and Manyverse, they could be a good alternative. The perfect solution, as for indigenous people, would be a sort of walkie-talkie, where verbal communication is the main medium. Imagine small bits of encrypted audio files being shared thru SSB
I finally learned how to build the latest LibreRouter firmware, which comes bundled with Pirania and FirstBootWizard. Unfortunately
shared-state which is responsible for gossiping pieces of data around the mesh nodes still has some bugs when working with Pirania, so the captive-portal has been set off for past weeks. I'll get around to debugging it together with @markitos soon.
The network has been acting very weird lately, and thru my tests I believe the wind may be the cause. So we've started putting up a cement pole to stabilize the hill backbone node. Waiting for a neighbor heal from a cold for us to finish it, pictures soon.
Been a while I last did an update. The past weeks have been pretty exciting for us here in Moinho with the visit of @markitos. Moinho is now the test bed for some of the most awaited softwares in the #communitynetworks scene, that me and Marcos worked full time for over 2 weeks, while rain pored outside.
We've been working on FBW since my last visit to #quintanalibre for the Summit. I've talked a bit about it here, but it's basically the easiest way possible for anyone to get a mesh network started. Basically it scans nearby networks, lets the user choose which one is the network it wants the new router to join or creates a new one. The software then sucks the other routers configurations, and applies a custom name for the new router.
Marcos has taken it far from where I left off several months ago, and so we could put some finishing touches on it now. There are some advanced features we need to work on it, but the basics have been successfully tested.
We've built it as part of the #librerouter project, so it's going to be part of the firmware shipping with it. But it also works any router running #libremesh, as long as it has 5Ghz for now.
This has been my personal most awaited meshing software of all times. A captive-portal solution that works on the mesh has been long overdue, and it seems that my personal need has fueled the development and finally a beta software. Many were involved in building this including Marcos, Guido, Gio, @Nico Pace and @nicoechaniz. I've experimented with nodogsplash before, but it proved to be buggy within mesh networks.
While I last tackled last year, Marcos and Gio (core Retroshare and Libremesh dev) have been working on an amazing solution to share data amongst mesh nodes. With this new shared-state software, and the Pirania-hooks Marcos built for it, people can walk around the village and will have access with their vouchers anywhere where there's network.
A captive-portal has been such as big blocker for me because it seems like the best way to provide discoverability to the local apps I'd like to build using decentralized protocols and also to provide visitors with some relevant information about the village, as drawn in my year old vision for the mesh:
We contracted 3Mbs of bandwidth, which is 3x better then the current 1Mb we shared amongst almost 30 people in the mesh. My initial speed tests have showed 16Mbs download speed I don't expect that it'll stay that high, but it proves that they can deliver such speeds, which means we could provide internet for the whole village over our network.
Providing internet has never been my personal goal for this project, but it's what everyone in the village expects from it, as local content is still something unknown to them. So this is an important step for them, and also because it means the network can finally expand more, as we've been limited by the 1Mb satellite provided us with.
I've had the Rock64 laying around for a long time now, but haven't really put it to use. It was thinking about a community server and the Kayapo project that really got be hyped about working with it again.
I've been starting a bunch of repos such as community-server, community-server-ui and many more for apps that will live in this #community-server-ecosystem.
Guido and Marcos helped me setup ZeroTier and to point a public IPv4 to my community server. With that, anyone who visits moinho.digital will be directed to my local server How cool is that? Another huge advantage with that, is that now the server has become a point of entry to my local network, so Marcos and other devs can
ssh into any of my nodes to help out with maintenance and to check how the beta softwares are running.
I'll spend the next months working on this ecosystem of server/apps for both Moinho and the Kayapo project. So I'll probably write a detailed post later on going more in depth of how it should work and the reasons behind it.
We have definitely thought about it @Christian Bundy, although we're not so sure where we could get a connection from. I'll take a look, our small backbone tower desperately needs some upgrading for it to resist the rainy season.
Do you have any plans to put up your own tower? I've been talking to Florian from Growing Tower and he's been working on a building technique that lets you build strong towers without ladders or cranes. Maybe useful? https://youtu.be/brrANrfcFk4
That's good to know @Nico Pace.
@sam_uk, I've got a Rock64 laying around exactly for things like that. On the last Community Networks Summit we got together and @gmarcos87 installed pi-hole on his Pi, haven't heard about it since.
Would doing QoS on the routers be a good idea so that the network doesn't get clogged by videos (Youtube, Nextflix)?
That's what we currently have @sam_uk, but doesn't scale at all.
Looked into how to connect to fiber in town, and it seems such thing doesn't exist. Both providers we have bring broadband from about 200Km away thru radio. So our dreams of connecting Moinho to fiber with our own infrastructure aren't possible yet.
The mesh had been on hold for about a month while I travelled. A few days ago things started happening out of a sudden.
First, without any help my neighbor Aridi was able to bring the network back alive. I was pretty surprised and very happy that things are starting to decentralize from my being.
Second, while I was walking back to Moinho I noticed that people were working on a tower that had been up for a while. It was the guys from a new provider that worked with a fiber connection, and they were putting up antennas directed at Moinho, with a clear vision to our backbone node.
They weren't very friendly, specially the boss. Answered a few questions, and didn't like it when I talked about sharing broadband. It's a great opportunity to get a better broadband service, but while talking online with the boss he warned that they would block the service if they found out it was being shared. He has been overall a big ass hole, and hasn't shown any sign of wanting to cooperate with our network.
I got in touch with the #communitynetworks organizations to figure out the situation. Guys from #coolab here in Brasil pointed me to a few legislations and said it's illegal for them to block service. @nicoechaniz also offered me some really good advice, and pointed out that the most sustainable solution would be for us to make our own link to town, and buy fiber directly.
Me and Aridi talked about the possibilities and decided to buy a 10mb residential plan and share it in our network to start off, despite the warning. We also decided to start investigating on what's needed to get our own infrastructure and how to finance it. I was surprised that Aridi suggested #micro-credit as a way, and I think it could work.
This is the current map of our nodes:
We currently have 6 nodes, serving 8 homes with two 1Mb satellite connections, which is terrible quality. With the new broadband service we have the possibility to expand to the whole village but we could have our service blocked for sharing. The legal road would be to buy a dedicated service, which is about 3x the price of the residential plan, and get a license. But I'm still investigating the legislation to understand it well.
I'm thinking on experimenting with some community methods for economic organization even without the proper tech, starting out on WhatsApp groups. That'll buy us time until we can get a proper captive-portal solution working on the mesh, to make the discovery of the applications possible; and to actually conceptualize, design and develop the applications as well.
The first method I'd like to test out is the one shared by @Modante here, on creating and managing a community #timebank with WhatsApp groups. Such experiments should serve to test out the methods, so that we can refine them into #community-first applications.
I remember well when you thought me the relation between gain and the pigtail size on the rooftop with the LibreRouter @gmarcos87, but thanks for the remimder
Unfortunately we only have these 1m long pigtails for now. Hopefully we'll acquire the proper tools to make our own in the future.
@luandro, I love reading about this project – it's very inspirational!
This year has been by very very far the most rainy of all rainy seasons I've spent here in Moinho. That's great, since there has been a lot of rivers and springs drying out all around the Chapada dos Veadeiros region (I'd guess it's a world phenomenon).
But all this has proven to be a challenge for our network. We're still in the beginning of the rainy season, and we've already lost one router (WDR 3500) in my house because of lightning and three router power supplies around the village due to water on their circuits.
So to counter the power supply problem, my partner in networking and Moinho native, Aridi, created this high-tech gig:
Basically a bottle cut in half, with the power-supply inside, sealed with silicon where the wires go thru. It's been on use for some time and taken a lot of constant rain, and we haven't had a circuited power supply since.
A while ago we installed our retracting pole gig on the hill node, which serves as bridge to most of the village:
It's concreted about 50cm bellow the ground, and is about 3m high. It also shrinks to about 1.5m for maintenance. It's been up for about a month and has been a great success.
With a firm and stable backbone we were able to install one more node in one of the main streets of the village, where a lot of people have shown interest on joining.
With this new installation we have secured nodes in main points of the village, making expansion now easier. But for us to expand we now definitely need a captive-portal solution to control access, and provide a door for community interactions and content, otherwise there's no incentive to join our network for those who already have broadband.
We, from the #libremesh community, have been trying to build our own solution that's more adequate for mesh networks then nodogsplash, which was pitbull and just recently rebranded to piranha. @nicoechaniz and @Nico Pace started working on it, then me and @gmarcos87 tried finishing. We got to a point that it worked pretty good within a node, but didn't work on other nodes in the network. @Hiure who's also very excited to get this working, has been helping us a lot with testing. He went to Europe for the Internet Governance Forum and is now visiting communities such as #freifunk in Germany and Aurea Social in Barcelona. He got some experts to help us out, and found out that we have a problem with the layer 3 software we use in our stack:
bmx. Quick solutions would be to discard layer 3 meshing or to use babel instead. But the Libremesh stack is pretty solidified and I believe such a change isn't so easy to make.
So we've decided to take a few steps back and go back to NoDogSplash in order to begin experimentations. I'll keep working on the same repo, but now on the nodogsplash folder, on scripts to easily setup and administrate the captive-portal and the voucher system. I had most of it working a few months ago, but my installation script has been crashing my network on my last tries. I'll investigate it further with the #coolab and #altermundi gangs and hopefully we'll have something decent working soon enough.
This has been going on for 7 months, and it still hasn't achieved the initial goals of enabling easier on-boarding for local communities. It's a big challenge since most people don't have any connectivity, so the first quest has been for a way of achieving that in a more decentralized/distributed manner, which got me to #mesh-networking and the beautiful work on #communitynetworks around the .
Once people gain access to the internet, the next big challenge is for discovery, as people usually only stay on the Facebook/Google islands. The captive-portal will serve as a door for community content thru client applications running on #distributedtech such as SSB and distributed hash tables; with a single and meaningful purpose for the community as outlined in #open-app-ecosystem; and running on a mesh infrastructure making it resilient even if the internet itself goes down.
We've been trying to solve this last problem of discovery so we can move on to the next stage which is conceptualizing and building these meaningful community apps, and where the real on-boarding of locals from the community and the real fun starts.
The adventure continues...
The mesh has been down for the past few weeks. Rainy season started way earlier then expected and our wooden pole setup wasn't handling the winds. So I finally got some iron poles that we cemented to the ground. A heavy rain started the moment we finished pouring the cement, and we never went back there, since my neighbor has been traveling since. I'm hoping that it didn't turn into a total mess and is now well held.
It's been raining a lot, which has held me back a bit, but I feel it's time to get back to the field and re-structure the mesh with the new equipment we got paid by some neighbors who want to get in. This is my new plan based on our resources and points of interest:
The only nodes in operation are Aridi's and Bete's, which are connected to each other, and have been even before the mesh started. I've introduced Aridi before, he's been my helping hand so far, but now he got a steady job outside of Moinho and won't be able to help me as regularly. Luckily Felix, who lives right below the hill back-bone, has been getting pumped about the network and has been helping me more, as he's connection will depend on it. He's the one traveling, but should be arriving any moment, and we'll get back to action.
I've been helping Susi to try and get internet for a while. She moved to a new home, and would very much get into our network and has even paid for her node already. It's a big challenge since she's a bit isolated from the rest of the nodes, but her house is in a point of great interest, because it's the entrance to the village. Leo, who is who introduced and welcomed me to Moinho and also has been helping me build my house, is going to start with a satellite plan. Now I just have to convince them to share with each other, and convince the Hostel to let me put a node to act as a bridge for them.
Eddy's bar seems to be the only home in that street that we see from up the hill. His brother who lives next door, and some of his other neighbors have also shown interest in joining the network. So it would be a perfect point to connect to.
I've been working a lot to get the captive-portal working in order to better manage the whole economics around broadband. But have been hitting on a few problems that I'm trying to figure out with the guys from #altermundi. After we get it working I'll focus back the #open-app-ecosystem as we'll have a point of entry for the ecosystem. I'm eager to get something working with #valueflows and at least a simple #cabal fork that has better UI for group discussions then Whatsapp, which has been a big problem for neighbors who are trying to organize themselves around a water issue in the village.
That's it for today, I hope next post will have pics of the actual nodes being setup and everyone getting connected!
Finally back online after dwelling with electricity
This weeks lesson was on how not to do things. The night I arrived back home I noticed the network was working. I hadn't payed the internet bill hoping I could negotiate it at a lower price, so I got cut off. But with the mesh I could start using my neighbors connection until I managed to pay my bill.
The next morning the network was offline and I was all pumped coming back from the summit and hanging out with all the hacky guys from Alter Mundi and meeting the really awesome guys from #coolab, @Troian and @Hiure (who is who went the Wapichan mesh gig @noffle posted about and I dreamed so much with). So I decided to go up the hill to check the network at the backbone nodes who connect me to the center of the village and also serves a access point to two families who live there. That's when things started going bad.
When I got to a first house, that hadn't visited before and was installed by my neighbor Aridi, I couldn't get any decent read from any AP, so I brought the node down. Next I went to the very top where the backbone nodes are, and I couldn't simply reset them or tell if they were turned on. And so it took me a while to unbox them and see we didn't have power. I took both the routers down, went to the house where power came from and it was badly plugged to the wall
The sun was unusually hot, and it's a long and steep walk to the top. I'm not sure what happened when I got there that I decided to test my electrical skills, which had been just acquired by getting my home's solar setup working. So I chopped off the wires that connect the router's power directly to the house by a 70m cable. Went back down, updated the routers to a new OpenWRT v18 build we're testing, and back up...
The sun was again hot and the walk steep and the routers at a terrible place to work. I rejoined the router's power supply to the cable with a hight fusion tape, and screamed so that they would connect the power to the wall, it seemed to work at first, so I put them back in the box and re-tied them to the wooden pole with wire, really badly as I couldn't take the heat anymore.
When I got back to the home they said it all went
Long story short, just last night I was able to finally go back up to the hill with Aridi and find the terrible short-circuit I had left when joining the wires. We fixed it and everything is back up.
It wasn't in vain. I learned better wire joining after short circuiting my solar setup a few times before getting it right with the help of a friend. And am now losing fear of electricity and starting to respect it.
After the Latin American Community Networks Summit @nicoechaniz invited me to join @gmarcos87 and Santiago (a guy working on building satellites) for a brief hackaton in Quintana. Our goals were to work on software that will be built-in to #librerouter as it's premiere gets closer.
San and Nico were able to compile new Libre Mesh firmwares directly from OpenWRT, which enabled us to build with it's latest versions, and become less dependent on the lime-sdk which has only one maintainer.
In the mean time, me and Marcos were working on the FirstBootWizard which is basically the easiest possible way to start a mesh with a fresh LibreMesh node. It scans around and tries to find other nodes, in case it finds em, the user can select which node in the mesh is wants the new node to copy configs from. In case no nodes are found it gives the user the ability to setup new configs directly from the Lime App, which is a user friendly application that comes bundled with LibreMesh.
After we got the basics of FirstBootWizard working, we worked for my last few hours in Quintana in the captive-portal software Pitbull, which is in the process of being rebranded to Piranha, as it enables a swarm of packet eating nodes. This was probably the most requested software from the community networks that were present at the summit, and as I've mentioned a few times before, it's a must for our mesh in Moinho to scale. It serves a few different purposes:
- Shows a captive-portal to newly connected users, which is necessary for a community to promote local content, and in the case of Moinho, also to promote local cultural spots to visitors in a map
- A voucher system so that neighbors can share cost of bandwidth
- A Quality of Service system with traffic shaping to stop non-payers from sucking all broadband, but still letting those who can't pay have a bit of connectivity
We already had some parts of it working from the last time I was in Quintana. This time, although we did very little work on it, we were able to get a better idea of it's structure and what's missing for it to function as it should. Hopefully we can continue working on it remotely.
With these new softwares the Mesh Kit tutorial I proposed starts to become meaningless, as creating a mesh is becoming very intuitive thru FirstBootWizard and Lime App. So in a very near future starting a mesh will be as easy as downloading the LibreMesh firmware, connecting to your node and going thru a few easy steps to customize the network to your communities needs
Happy meshing everyone
Next step is to lift the Open App Hub from prototype level. This includes getting SSB to work properly with the UI; updating GraphQL and GraphqQL web-socket transport; and building the first apps for the ecosystem: a cabal-chat made for communities and the simplest example of a #valueflows app to get us started with the power of economic apps.
For the past few days I've been observing thru WhatsApp the people in Moinho having serious discussions about the use of our watershed, and it's become a full mess with people taking some opinions as personal attacks. While in the round discussions in the summit, I came to the conclusion that UI's are also political forms of expression. So designing a simple chat app, with a UI/UX that permits proper discussions within a community might help enabling better organizing.
On the airport once again, this time an expected surprise. @nicoechaniz invited me last week for the Community Networks Summit happening on Buenos Aires tomorrow, and for a hackaton in #quintanalibre happening after the event. We'll be working on #librerouter related stuff, hopefully getting the pitbull broadband sharing system working. Much more then that, like my last trip to Quintana I expect that it'll be a life changing experience, and will make the development of our network in Moinho give some quantum leaps.
For the past few weeks, since my last update, I've been working on getting the application seed ready so we can actually start developing some applications for the ecosystem. I abandoned the last repo which used react-native-web and started a new one from scratch using react-native-dom which has been working beautifully for building universal applications. I've also started working on app-hub-desktop for the web apps to run from.
I'm really excited for the development happening around #scuttle-shell since it's really necessary for the ecosystem to work.
I've also started working on the GraphQL plugin for cabal-core so we can use it with the open-app-graphql-server and have an universal chat app, which is the ideal application to start off the ecosystem. Hope to get at least a working prototype in the next few days so we can test it out on the hackaton in Quintana.
Time to get onboard, cheers and love u all
So far so good @DarkDrgn2k, but haven't really started messing with it yet. Soon I hope...