Overview: Tiny Oracle was an idea I had a few months ago. Its based around aggregating new / traffic / weather / tweets etc into one place, and then generating how a city “feels” based on that data. I didn’t get to the data part this week, but I did make some good progress with the hardware, and even started writing my own restAPI to handle requests. The results, despite it just being a tech test were whimsical.
System: This took a bit of a twisty road. Basically I started thinking I would make a Python based bot that just aggregated all my feed data, parse it, then toss some commands over to a BaaS provider like PubNub or AdafruitIO. But I once again ran into the issue w/ the Feather and its MQTT library, that there’s something buried in it, that prevents it from being totally non-blocking. I knew the particle library didn’t have this issue, so I started digging around in it, and seeing if I could re-mod the mod to re-use for the Feather. But it looked like the particle library was just doing GET requests. So I started looking around for some info on REST APIs, and decided I didn’t need a Pub/Sub setup. I found some tutorials, downloaded Flask (which I like and am familiar with), and starting making a basic API to pass on some JSON. The Feather makes a request on a timer, grabs the JSON the server serves up, and then prints out a message and does a little display based on it. I found it pretty straight forward to build this little Server / Client system. It doesn’t have any security, so its only for local testing, but I would like to make a version that does. But I would like to keep rolling my own solutions going forward.
Things I Experimented With: FLASK rest-ful, requests, protocols.
Things I Learned: A lot about basic rest apis! And that you really don’t have to send things to “my butt” if it doesn’t have to go to there. Most of the MQTT libraries for the various BaaS providers seem to have some kind of blocking issue (pubnub / adafruit) in the subscribe function. Also that a lot of data can still just be grabbed with just GET requests.
Future Iterations: I really want to get this to a point where its parsing different data feeds to build its feelings. I also want to keep building a little restAPI for more of my personal projects.
Overview: For this week, my goal was to poke around with Wireshark and learn a bit more about networks. I decided I would also try using pyshark, which is a python wrapper around Wireshark’s command-line utility tshark. Originally I had thought I would pipe some data into Processing to do some viz with it, but at the end of the week realized that Processing already had Carnivore as a library, which allowed packet for in-Porocessing packet sniffing. Wish I’d thought of looking to see if that existed earlier. But oh well? Its nice to know it exists, and I will explore it in the future.
What I ended up doing instead was writing some utility functions to comb through captured data and return the most active of whatever items you send through it. Pyshark has some good ways to dig down into packets, but it can also just grabs all the summaries. So if you want to compare traffic, look for protocols, or just poke around, you can toss together some basic functions to do so.
I think in the end I liked doing the utility functions more than the idea of visualizing it. I would get to the point of having a dataset, and instead of moving on, would go back and dig around in the packets a bit more.
Of Note: One thing to note though is that LiveCapture in pyshark can sometimes be borked in OSX. Wireshark (and other sniffers) require read access to “/dev/bpf*” which is usually owned by root/wheel. When you use Carnivore, you have to change the permissions on this to 777, which is…very permissive. You will have to do this each time. Wireshark creates a user group, and adds the person who installed wireshark to this group. So pyshark should be able to use tshark under your user without this extra step.
If you can’t get it working then either you need different permissions, or like me, you are trying to read a networking interface that doesn’t exist…because you weren’t paying attention. Then ended up in a 1 day rabbit hole…but I learned some things about what’s depreciated in the OSX system.
Other things I rediscovered: using man pages, and using python’s dir() function. Which is really useful when you want to know what kind of methods you can use on something.