Though I actually wrote this a couple years ago I finally decided to contribute this utility back to the community. This tool allows a graphical remote connection to a MythTV database and can copy and delete recordings over a Samba share. It is a graphical utility written in Java to search recordings, see details of recordings, copy single and batches of recordings, and other features for remote access to recorded programs. This should work on Windows, Mac, and Linux. Hopefully other people can get as much use out of this tool as I have. Feel free to post a comment if you found this useful.



MythAssistant.jar

MythAssistant.exe

MythAssistant-source.jar

In Java, Beans are an important part of good programming practices and good Object-Oriented coding. There has been a lot of discussion recently about monitoring changes on beans other such functionality that is not inherently built into Java. With my Magic Beans project I've already done a lot of things along these lines, but recently I've been thinking more and more about how powerful beans could be if they were to extend outside the boundaries of the norm. What if you could deal with Beans like you do with Connections in SQL? What if you could create a transaction that would give you a new copy of a Bean and when you are done with it you can "commit" that bean back to the original? What if you had transactional monitoring of beans that goes beyond the normal Observer/Observable concepts?

I keep coming back to beans as one of my biggest stumbling blocks for writing good and efficient code. Inevitably I'm going to have to revisit Magic Beans and finally create the end-all-be-all for Bean handling.

A co-worker and I were discussing the possibilities of performance gain doing extraction of ZIP files using multiple threads rather than the typically single-threading extraction that the majority (if not all) mainstream archive extraction utilities use and given Java's great ZIP file support built-in it seemed rather trivial to give this a shot, thus, UberZip was born.

UberZip is my simple little sample Java command-line application to extract files where you can specify the number of threads to utilize during extraction. My #1 biggest headache is extracting Eclipse from the ZIP file you get, so I decided that would be an ideal test of my little program. I downloaded the Eclipse J2EE bundle (3.1.1), which is a happy 132 meg with thousands of files.

I need to test further with a machine that actually can better utilize multiple threads, but here are the stats for my AMD Athlon 64 3200+ (Hyperthreaded, so I actually get a very slight benefit) running on Windows XP:

14.7 seconds with 1 thread

13.4 seconds with 5 threads

11.6 seconds with 30 threads

Anything above 30 threads seems to actually create more overhead than gain.

Now, to be fair I matched this up against the fastest unzipper I am aware of, 7zip (http://www.7-zip.org). It took right at about 14 seconds to unzip the file inside 7zip, which seems to fall nearly perfectly in-line with the single-threaded execution of UberZip.

For those of you that would be interested in taking a look at this very simple example, I have committed the source to my public repository:

svn://captiveimagination.com/public/uberzip/trunk

Further, if you'd simply rather download the JAR or EXE I have uploaded copies of it for those of you that would like to unzip files "uber" fast. :)

uberzip.exe

uberzip.jar

After taking this to try out on my work machine (Dual Xeon dual-core 3.2 GHz processors + 4 gig of memory = 8 processors in Windows) I got the following stats:

7zip - 25 seconds
UberZip (1 thread) - 22.17 seconds
UberZip (5 threads) - 19.5 seconds
UberZip (30 threads) - 21.6 seconds

Oddly it would seem about 5 threads is the "sweet-spot" for this machine. Though some of the results are a bit strange and it would seem on this machine the hard drives don't perform quite as well as my home machine it is obvious that at some level of configuration multithreading you can gain some good performance on the single-threaded applications out there.

Perhaps someone else will find this information useful and turn UberZip into a product people can use. ;)

Update (2017.05.31)

I've completely re-written this functionality in Scala and posted it on GitHub: https://github.com/outr/uberzip.  It's faster than ever and more cleanly written.  Tested on the latest Eclipse ZIP (320 meg) it can unzip 2,981 files in 0.73 seconds. Doing the same test on the same machine with Linux unzip took 1.7 seconds.

It's been a while since I posted and after dealing with this horrid bug in Flash I figured it would be a great topic to get me back on track.

So, apparently there's this nifty little bug in Flash/ActionScript with the URLRequest object when you attempt to do a file upload, it uses a completely different browser session than the web browser or any other requests made from within Flash (image loading, sound loading, resource loading, etc.). This isn't a problem unless you care anything about security...and unfortunately most of us do. So the project I'm currently working on uses JCIFS (http://jcifs.samba.org/) for SSO, which is actually very nice by the way, and it uses a servlet filter configured in the web.xml to authenticate the user before they ever hit the page, so by the time they reach the Servlet they've properly been authenticated and I don't have to worry about rogue users.

Okay, so that all works well and good until I try to do a file upload. I try to establish a URLRequest to my JSLServlet (of course this is jSeamless) that I use for everything else, but this time I get a nifty little login prompt using basic authentication. Now, this is of course bad, but what makes it even worse is that Flash simply prompts you and then disregards it and streams the upload into the outer darkness (where there is weeping and gnashing of teeth) so the upload from the Flash perspective worked and uploaded fully (note, before I even enter anything into the login prompt) but the server never receives the upload at all. If I enter my credentials it accepts it and then disappears, but really doesn't care anything about it.

Okay, now for the hack. Now, for you programmers that are faint of heart, don't try this at home. For the rest of us that have to do the job no matter how dirty it might get, please forgive me for what follows...if there were another way I would have done it.

Alright, so I create a new Servlet filter called UploadOverrideFilter (yes, you see where this is going) that gets put in the filter order before the NtmlHttpFilter (JCIFS filter for SSO) that detects if the reference is made to an upload (endswith("jslupload") in my case). Now, what took me a bit of work to figure out is that all mechanism for referencing directly to the Servlet that the filter can pass off to is either barred or deprecated (from ServletContext there are methods for getting a Servlet, but they always return null since deprecation) so I pull out my ugly stick and turn JSLServlet into an evil beast by maintaining a static reference to "instance" that is the initted reference to the currently context JSLServlet instance. Then I simply reference that static JSLServlet instance and explicitly call doPost(...) instead of passing off to the filter chain. See the source code for yourself:

public class UploadOverrideFilter implements Filter {
 public void init(FilterConfig config) {
 }

 public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) throws IOException, ServletException {
  if (((HttpServletRequest)request).getRequestURI().endsWith("jslupload")) {
   JSLServlet.instance.doPost((HttpServletRequest)request, (HttpServletResponse)response);
  } else {
   chain.doFilter(request, response);
  }
 }

 public void destroy() {
 }
}

Yep, it's having to write code like this that will be the death of me...a very good reason why I prefer to stick with 100% Java whenever possible is so I don't have to write hacks like this.

Well, I guess it wasn't as bad as it could have been, but it's still a hack and it's things like this that keep me up at night in the cold sweats.

So I lied, apparently my hackery wasn't enough to appease IE even though Firefox happily worked going forward. Apparently in IE it determines, "Oh, I've been authenticated on this URL before...so lets send broken authentication information instead of an upload", and then no matter what you respond back to the request with it just ignores it an on the Flash side acts like it sends the file (very quickly), but never actually comes across.

So now we must expand the previous hackery to not only ignore authentication on uploads, but we have to send to a different "url" in order to keep Flash from thinking it needs to authenticate. I originally tried doing a variation on the directory structure (ie. /authenticated and /unauthenticated), but it's a tricksy URLRequest and decided that it was the same host so it really wanted to send broken authentication information.

My solution? Make Tomcat listen on another port by adding the following line to my server.xml:

<Connector connectionTimeout="20000" maxThreads="150" port="15000" protocol="HTTP/1.1" redirectPort="8443"/>

Then I told my upload to send to the exact same URL except on a different port. I still had to have my security hackery to keep it from requesting authentication, but that did the trick. Now both IE and Firefox can upload files in a JCIFS protected container and the client is none the wiser...unless of course a firewall is blocking that port for which they are trying to upload to....but such are the hacks we must work with when dealing with Flash.

On a side note I will say that Flex 3 has functionality built-in (apparently they got enough rocks through their windows from 2.01 to make them realize they needed to fix it) to tell it to handle authentication or not via a boolean flag. None of this would have been necessary if they would have had the fore-sight to implement that in the first place, but alas, mine is not to reason why...

My massive hack for native drag-and-drop in jSeamless Flex implementation is functional, but only in Firefox. Apparently there is an interesting "feature" in IE that even though you may sign your Applet and the user grants that they way to trust said Applet, it still never gives you read access to the file system.

That being said, here is my outlined approach for another hack that avoids the use of an Applet in favor of Java Web Start. I know Java Web Start can do what I want to do, but to my knowledge there is no way to communicate directly between the client-side web site and the JWS instance.

1. Load the JWS application that stays hidden


2. Load the Flex content page


3. Listen via JavaScript to the mouse cursor entering the browser window - throw an event to the server upon entry


4. Server sends event back to JWS (either socket or delayed Ajax necessary for both JWS and Flex)


5. JWS application triggers the single-pixel-at-mouse-position-window that is listening for dragging


6. If JWS doesn't receive a drag enter event it goes back to sleep and hides our friendly, yet scary, window


7. If a drag enter is thrown it continues to follow the mouse until either the mouse exits the window (another event thrown by JavaScript to Flex to the Server and back to JWS) or a drop event is triggered in the window.


8. If JWS receives a window exit event it goes back to sleep.


9. If JWS receives the drop event a new event is sent down to the server with the list of files dropped.


10. Server sends an event to Flex requesting the current component the mouse is over and temporarily stores the list of files dropped via a request id.


11. Flex determines the current component the mouse is over and sends an event back to the server with that id.


12. Server triggers internal jSeamless event for file dnd with the list of files and the component id (if the component accepts the drop).


13. Upon server-side accept of a file an event is sent to JWS to stream the file


14. JWS receives file stream request and streams the file to the server.

How's that for evil? I thought the original hack was bad, but this is just downright insane. However, it would seem this is the only plausible way to get native drag and drop support.

I had made the commitment a while back that I was going to add complete and perfect drag-and-drop support to jSeamless. The Flex Implementation proclaims support for DnD so I've just pushed off the investigation up until recently. However, now that I've come to implement this great functionality I realize that there is absolutely no support for native DnD. What I refer to as "native DnD" is the ability to drag a file or any other "draggable" content from outside of the application into the application and allow it to pick up and use that content.

Okay, so I figure I'll just do the same type of JavaScript hacks I've done all along to get missing functionality into Flash. To my surprise HTML/JavaScript has no native DnD support either. At this point I'm not only surprised, but afraid my dream of a beautiful drag-and-drop solution for jSeamless is in danger of never reaching fruition.

So I spent the day brainstorming with a colleague about how this can be done. Honestly, there was never a viable solution that wasn't a massive hack, and what we ended up with was actually far better than expected. The solution consisted of using a signed Java Applet to display a single pixel JWindow linked to the cursor when the cursor enters the browser window, it is able to use this as a check to see if the cursor is dragging a droppable item. If it is not carrying something into the browser it then disables itself until the cursor leaves the browser and re-enters again. However, if it is carrying something it remains locked to the tip of the cursor and accepts the droppable wherever they drop it in the application. This JWindow intercepts the drop event, passes an event to JavaScript, JavaScript passes an event to the Flash content and is handled appropriately from there. The event is assigned an ID that can be referenced back from the Flash, to the JavaScript, to the Applet in order to initiate the download at any point in the future.

Yes, this is a massive hack, but does provide a solution that is quite seamless to the user apart from the nasty, "Would you like to grant access to this application to do whatever it wants to your hard drive?" (okay, so it's a little friendlier than that) prompt that must be displayed for the Applet to be able to know pointer location and be able to actually read the file contents from the hard drive.

Fortunately internal drag-and-drop is quite easy to write into Flex. If all goes well, this will all be finished and useful in jSeamless next week. However, for reasons of the aforementioned "evil prompt" it will be disabled by default and will have to be specified to be turned on to be used.

Today was massive hack day. ;)