Distributed Analytics Project – Progress Report

So, I made the hub, got the client to download the analytics-urls list, got the client to connect to one of those urls… but them… I figured that I would need to either: upload the whole file to get analyzed OR write a multiple-state script that is really going to complicate my life.

Today I thought about using python script and then evaluating at the client side but there is no safe way of doing that.

Uploading the file isn’t really practical because I plan to analyze huge files in the future.

So in all, there are three options:
Client sends file to Analytics-Page, receive report and then forwards to HubClient sends url and size of file to Analytics-Page, executes custom script received, sends result to Hub
Client analyzes file, report to Hub

The first isn’t scalable, the second is too complex(custom script) or prone to attacks(python’s eval()).
That leaves removing the analytics page and doing everything at the Client level.
My code would be revealed and someone could simply edit out the part that sends me the analysis result. I believe they won’t  because:- I will ask them politely not to
– I will format their results and make it easy to check them on the Hub side, plus they get to share/advertise the file/url analyzed

Lots of stuff to think about and code.

Leave a comment

Filed under Uncategorized

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s