On the Gravy Train to the Super Bowl

It has been awhile since my last post, and this space is formally commandeered for my Tailgate Adventures. Each Seahawk home game I party and grill with friends before the game. Each Sunday I try to come up with a new and unique menu in a vain and hopeless quest for the coveted Mud Bone trophy. An award bestowed on the tailgate team with the best culinary talent.

This is opening weekend and I am not 100% sure my menu. I've settled on a Curried Pork Loin with a Jalapeno-Mango Chutney as the main dish with a side of home made Potato Wedges. No idea for a dessert yet, and I'm not completely happy with the menu but need to get the ingredients tonight. We'll see how it goes.

On a better front, the Amazing Jonathan combined forces with Lunchbox to restyle and paint our tailgate grill to match our Football and Tailgate teams. It is simply Awesome. Expect pictures soon (I just need to take them and figure how to attach them). This grill is now the center piece of our party efforts!

Drifting through data....

Today was pretty cool. It snowed, it was sunny, it snowed again, and then it rained. I guess a day that doesn't know what time it is shouldn't be expected to make up its mind on the weather....

Eve:

I wrote a query that gives us the average price for an item but only sorts the lowest X sell orders and the highest X buy orders. In the case of Zydrine I was looking for the cheapest 4M ISK worth for sale or 2M units whatever came first then the highest 2M units being bought. This query tells us that in general it is best to get your Zydrine in Metropolis and sell it in Tash-Murkon. What it does not tell us is how many units move through that region per day. 

The output is still very rough, I hope to learn how to make a python GUI sometime in the future. Plus we could have this in some sort of task that takes market snapshots every few hours.

All this data mining is making me feel like I'm drifting into the future. Very cool 

New week... New goals...

Last week trudged by and gouged a deep trench of bone chilling tired into my life. The major system upgrade at work is done (with plenty of little issues left to bug me) and I was also able to get the Trade Finder working. Some highlights include, learning how to index a table, adding error handling to the python script, and seeing the damn thing actually find some decent trades.

Currently the Trade Finder exists as two separate scripts. The Order Importer scours an email account every three minutes and grabs all emails from Eve-Central. These are examined and inserted into the MySQL database along with a few extra fields for if the order is in hisec, the time it was imported, the total value of a buy order and a UID.

The second script is the Trade Finder. Right now I have to hard-code user variables but it searches all of HiSec for the best trades based on the users cargo capacity, capital investment, start location, minimum profit, and maximum number of jumps. The output is an ugly DOS like window with no sorting but it runs in under 20 seconds and finds deals.

The future with contain me learning how to create a python GUI and find a way to host this for my friends to use. There is also questions about creating our own order importer. I'd like to use the EBank method of cache scraping but don't know if it is possible.

Slogging through the numbers

Sunday morning I found a better way to calculate the distances but creating a graph of only Hi Sec. This leaves about 500k routes to find. Started the script around noon yesterday and it has found 305k routes starting from 420 solar systems. My guess is that it will take another 50 hours because the algorithm is less efficient near the end.

More good news is that I logged onto Eve last night. My character still exists and retained about 130M ISK he had, plus a ton of items I need to think about selling. When I have time I may work towards the Crane -Blockade Runner. Need to get Advanced Piloting for that.

On other news this week is the big software upgrade. We are pretty ready but I still worry. We'll see how Tuesday's overnight goes.

Too much coding, not enough beer...

Finished the Dijkstra script about an hour ago. The database was trickier than expected (of course) so it is good I didn't stay up and try to hack it. And lucky for me that I have a well trained Phil that still answers when he sees my number. I suggest everyone out there in internets world gets their own Phil.

So tables differ from spreadsheets (duh, but stay with me). Building a table of distances in a spreadsheet would be a matter of X rows and X columns, one each for a location. You could find the distance A->B by looking in Column B of Row A. When it comes to SQL, computers like rows much better than columns. Something to do with the linear nature of seraching and itemization or something. The end result is that MySQL does not support 5,000 columns and would rather like three columns and 25+ million rows. 

The columns are Source, Destination, Number of Jumps. And there will be a row for each Source/Dest combination. (Here we can get cute and since the distance is the same in eaither direction we can keep the table half as long, as 12.5 million rows.)

I also optimized the script a bit by only using HiSec systems as starting points. There will still be routes to LoSec where these neighbor HiSec, but I gave these routes an extra 1,000 jump weight per LoSec. This should keep them out of the way. Pretty much if you see a route that is over 1,000 jumps, ignore it.

I started the script about 20 minutes ago and it has found 40,000 routes and closed 8 solar systems. The further it gets, the quicker the searches (as the return route has been calculated) so I expect it to be done in 3 weeks. Yeah, that sounds horrible but we'll see if Phil can improve it?

Hot Damn! Bring on the space babes!

After hours of work and many many mistakes, I've finally conquered the Dijkstra algorithm.

About six hours ago I decided to tackle the route distances. I began by porting some of the Eve map databases over to my production database, and then worked on creating a Graph. That probably took up to two hours. The real bitch was working with the Dijkstra code lying on the internets.

Most references want me to use PriorityDictionaries so I spent an hour or so trying to figure them out. I had a ton of python editors full of gibberish. In the end I resorted to the wikipedia Dijkstra site (http://en.wikipedia.org/wiki/Dijkstra%27s_algorithm). They have a pseudo-code logical outline for a solution.

My version would run through once, find the starting location (as it should) and then die (as it shouldn't). I thought maybe I can't alter a variable that is running a while statement I'm in, so far too much time was wasted tyring to solve that. The real problem was a variable that wasn't being redefined. Pretty much after it found the starting point (distance = 0) it would then look for something closer, and since there isn't it would die.

A quick "cut and paste" later and the shell started spewing out distances. I only checked one (Jita to Sizamod) against the Battle Clinic site (http://www.battleclinic.com/eve_online/route_finder.php) but it came up accurate (15 jumps in Hi-Sec). So I count that as a win!!

With Dijkstra written I need to define a distance table. This will have a column for each solar system and a row for each solar system. Should be about 12.5M entries. I'm jazzed and want to do it now, but I should get some sleep. And who knows how long the DB part will take to code??

Night all..

Just got back from watching The Pharaoh Serket and the Stone of Fire. A bunch of us went to see Whitney's play at the Seattle Children's Theatre. It was very very good, with the sets and changes being some of the best parts. 

Market Analysis Tool

The database has 77k orders in it. I just ran the Trade Finder looking for deals worth more than 10M ISK assuming I'm flying a Badger and have 50M ISK to trade with. This would give me a 20% return on investment!

The script pared 298k deals in 2 1/2 minutes and found 626 transactions that made more than 10M ISK. Many of these were Isogen or Noxium but a few were for Capacitor Consoles, and there were worth in the 15M-17M range. Though I haven't checked to see if they are in low sec or very far apart.

End result? The finder works, it may take some time but it does complete in the less than 5 minutes I was aiming for. Will have to see how it holds up as the database gets larger. Adding a function to exclude orders older then X hours would help with day to day stuff. Plus I could have it auto find every so many minutes or just for those orders that get imported each time? That last bit would be very slick. Maybe maintain a priority database of orders that is updated every every few minutes? Then when hit by a user it would just parse this list and use the users data? 

Not sure how I would implement that. Would I have to hard code the capacity and a minimum ISK value? That would assume the user would not have more ISK or cargo capacity. Just some thoughts. In the end this could make it faster but might miss some trades?

This weekend I hope to work on Dijkstra. But tomorrow might end up being Rock Band...

(BTW I have enabled comments, didn't know I needed to)