# Runner Progress **URL:** \ **Category:** Ruby Bench\ **Created:** [January 10, 2014, 6:33am UTC](https://community.miniprofiler.com/t/runner-progress/208 "2014-01-10T06:33:22Z")\ **Posts on this page:** 15\ **Page:** 1
**Author:** ![sam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/sam/32/377_2.png) [@sam](https://community.miniprofiler.com/u/sam)\ **Post date:** [January 10, 2014, 6:33am UTC](https://community.miniprofiler.com/t/runner-progress/208/1 "2014-01-10T06:33:22Z")
The runner side of the project has been running a lot slower than the UI, but I do feel I have made some interesting project. I provisioned an “old” desktop I have to run the bench, it is running ubuntu 12.04.03 LTS (x64) this is in line with what most servers out there should be running. I also had to spend an inordinate amount of time getting [cpufreutisl](https://wiki.debian.org/HowTo/CpuFrequencyScaling) to play nicely. I originally though I could disable all CPU scaling in the BIOS but it just does not play nice. Instead I force the CPU into performance mode which give very **even** and consistent results. Multiple runs of the Discourse bench come back with results within 1ms on multiple runs. I also added a `-b` option (best of) to the Discourse bench so we can repeat tests multiple times and pick the best. I have got a very large number of ruby builds installed now and have decided to standardize on chruby. Initially, I was using rbenv, but rbenv performs just way too much magic and `rbenv rehash` was taking forever. * * * Next week I will see if I can script a huge run of the Discourse bench across all the builds so we can graph it. However in the mean time I have a project for @andypike , @dmathieu and the UI team. Recently, funny-falcon has come up with a pretty awesome optimisation that makes method caching in Ruby much faster. We are trying to gather information about how much it improves things. ko1 is open to merging it in once all the concerns are addressed. You can read about this here: [http://bugs.ruby-lang.org/issues/9262](http://bugs.ruby-lang.org/issues/9262) I think it would completely awesome if we could highlight the differences between before and after in a pretty bar chart generated from the UI side. It would open up a completely new usage pattern for bench ui, and is incredibly useful. There are multiple perf patches pending, providing great comparisons of the behavior before and after is key to getting stuff accepted. Here are there results of running ```auto # runs the discourse bench 5 times, 300 reqs per page per tested $ ruby script/bench.rb -b 5 -i 300 ``` ###BEFORE ```auto --- home: 50: 49 75: 50 90: 52 99: 114 categories: 50: 75 75: 77 90: 83 99: 149 home_admin: 50: 57 75: 59 90: 60 99: 127 topic: 50: 12 75: 13 90: 13 99: 83 categories_admin: 50: 88 75: 90 90: 96 99: 161 topic_admin: 50: 22 75: 23 90: 25 99: 99 timings: load_rails: 3029 ruby-version: 2.1.0-p-1 rss_kb: 253160 architecture: amd64 operatingsystem: Ubuntu kernelversion: 3.8.0 memorysize: 23.55 GB physicalprocessorcount: 1 processor0: Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz virtual: physical ``` ###AFTER: ```auto --- topic_admin: 50: 20 75: 22 90: 24 99: 30 topic: 50: 12 75: 13 90: 14 99: 37 home_admin: 50: 54 75: 55 90: 58 99: 121 categories_admin: 50: 80 75: 82 90: 89 99: 156 categories: 50: 69 75: 71 90: 98 99: 159 home: 50: 47 75: 48 90: 50 99: 105 timings: load_rails: 2970 ruby-version: 2.1.0-p-1 rss_kb: 255392 architecture: amd64 operatingsystem: Ubuntu kernelversion: 3.8.0 memorysize: 23.55 GB physicalprocessorcount: 1 processor0: Intel(R) Core(TM) i7 CPU 960 @ 3.20GHz virtual: physical ``` * * * Is there any way we can get this graphed on the UI in a bar chart so we can attach to the ticket. (note: 99 = 99th percentile, just as ab returns it) ---
**Author:** ![dmathieu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/dmathieu/32/972_2.png) [@dmathieu](https://community.miniprofiler.com/u/dmathieu)\ **Post date:** [January 10, 2014, 9:48am UTC](https://community.miniprofiler.com/t/runner-progress/208/2 "2014-01-10T09:48:45Z")
We’re not generating bar charts for now. Only lines, displaying the evolution on time. I guess the feature you would be looking for in the UI here would be to be able to compare two results, which would then make sense to generate a bar chart. I just created [an issue about that](https://github.com/ruby-bench/ruby-bench/issues/17). As for your comparison here, since there’s no production deployment for now, and this would just be creating a runner for only two data points, wouldn’t it be just easier to generate an image (which doesn’t needs the UI, only a spreadsheet software) with the graph? ---
**Author:** ![sam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/sam/32/377_2.png) [@sam](https://community.miniprofiler.com/u/sam)\ **Post date:** [January 10, 2014, 4:37pm UTC](https://community.miniprofiler.com/t/runner-progress/208/3 "2014-01-10T16:37:14Z")
sure, but the motivation here is to promote the project. Its not critical we can wait for a larger data set. ---
**Author:** ![lazywei](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/lazywei/32/964_2.png) [@lazywei](https://community.miniprofiler.com/u/lazywei)\ **Post date:** [March 14, 2014, 11:50am UTC](https://community.miniprofiler.com/t/runner-progress/208/4 "2014-03-14T11:50:50Z")
I’m wondering where is the runner’s code? I can only find [https://github.com/ruby-bench/ruby-bench](https://github.com/ruby-bench/ruby-bench), which seems to be only a web interface. I suppose there must be a runner that can run benchmark and send the result to ‘ruby-bench’, right? Thanks ---
**Author:** ![wpp](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/wpp/32/811_2.png) [@wpp](https://community.miniprofiler.com/u/wpp)\ **Post date:** [August 15, 2014, 8:08am UTC](https://community.miniprofiler.com/t/runner-progress/208/5 "2014-08-15T08:08:09Z")
@lazywei I’m sure you’ve figured it out by now but the runner @sam was talking about is available at: > Is someone working on this? As far as I can tell this functionality is not in ruby-bench yet. I would get started if thats fine with you guys. ---
**Author:** ![lazywei](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/lazywei/32/964_2.png) [@lazywei](https://community.miniprofiler.com/u/lazywei)\ **Post date:** [August 15, 2014, 9:13am UTC](https://community.miniprofiler.com/t/runner-progress/208/6 "2014-08-15T09:13:35Z")
@wpp, I started some works on here: [https://github.com/lazywei/ruby-bench-docker](https://github.com/lazywei/ruby-bench-docker) We tried to build some docker environments to run benchmarks. Perhaps you can integrate your work in to this? Thanks. ---
**Author:** ![sam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/sam/32/377_2.png) [@sam](https://community.miniprofiler.com/u/sam)\ **Post date:** [August 18, 2014, 5:46am UTC](https://community.miniprofiler.com/t/runner-progress/208/7 "2014-08-18T05:46:14Z")
Any chance we can see any results, I would be happy to run something on my bench box if you provide me with instructions. ---
**Author:** ![wpp](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/wpp/32/811_2.png) [@wpp](https://community.miniprofiler.com/u/wpp)\ **Post date:** [August 18, 2014, 9:11pm UTC](https://community.miniprofiler.com/t/runner-progress/208/8 "2014-08-18T21:11:49Z")
Sorry for getting back so late. I was motivated to get something up on the weekend but the app currently uses a few technologies I’m not very familiar with (haml, coffeescript, wisper and docker?). I was also trying to contact @dmathieu on twitter and the IRC channel because I had/have a couple of questions about the models and the overall state of the project, but couldn’t get in touch with him. I whipped up a new Rails app quickly to make a screenshot and give you an Idea of what I have in mind: [Imgur: The magic of the Internet](http://imgur.com/SO5JB5t) (Sorry can’t upload directly because I’m a new user). [Regarding your original post:](http://samsaffron.com/archive/2013/12/11/call-to-action-long-running-ruby-benchmark) > More hardware and a nice domain name would also be awesome. I got `ruby-bench.com` just in case (and of course I’ll transfer it). > First is information gathering, writing the scripts needed to collect all the historical data into a database of sorts. I think @tenderlove might have some scripts which he used to measure Rails perf (AdequateRecord, mentioned in one of his talks). Maybe we can get something from him? So long story short: I think this project is super-important and I’m eager to get something real up and running. Although I probably won’t have significant time till the upcoming weekend. ---
**Author:** ![lazywei](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/lazywei/32/964_2.png) [@lazywei](https://community.miniprofiler.com/u/lazywei)\ **Post date:** [August 19, 2014, 3:24am UTC](https://community.miniprofiler.com/t/runner-progress/208/9 "2014-08-19T03:24:04Z")
> [@wpp](#): > > I was motivated to get something up on the weekend but the app currently uses a few technologies I’m not very familiar with (haml, coffeescript, wisper and docker?).I was also trying to contact @dmathieu on twitter and the IRC channel because I had/have a couple of questions about the models and the overall state of the project,but couldn’t get in touch with him. You mean [ruby-bench](https://github.com/ruby-bench/ruby-bench "ruby-bench")? Please let me know if you get question on this Rails app, I might be able to answer some of them. > [@wpp](#): > > I whipped up a new Rails app quickly to make a screenshot and give you an Idea of what I have in mind:[Imgur: The magic of the Internet](http://imgur.com/SO5JB5t) (Sorry can’t upload directly because I’m a new user). Can we integrate this into original Rails app (ruby-bench)? ---
**Author:** ![wpp](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/wpp/32/811_2.png) [@wpp](https://community.miniprofiler.com/u/wpp)\ **Post date:** [August 19, 2014, 7:22am UTC](https://community.miniprofiler.com/t/runner-progress/208/10 "2014-08-19T07:22:11Z")
> Can we integrate this into original Rails app (ruby-bench)? Absolutely, I really don’t want to fork the project, it was and is just a “mock-up” app because I need to learn haml and coffee. Yes [ruby-bench](https://github.com/ruby-bench/ruby-bench). I’ll send you my questions when I get home, thanks! (@work now). Cheers ---
**Author:** ![dmathieu](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/dmathieu/32/972_2.png) [@dmathieu](https://community.miniprofiler.com/u/dmathieu)\ **Post date:** [August 19, 2014, 8:01am UTC](https://community.miniprofiler.com/t/runner-progress/208/11 "2014-08-19T08:01:50Z")
> [@wpp](#): > > I was also trying to contact @dmathieu on twitter and the IRC channel because I had/have a couple of questions about the models and the overall state of the project, Sorry about that. I completely missed your twitter mentions. We can completely integrate any improvement you’d want to make on the ruby-bench rails app. The current state of things is that @lazywei is finishing a Google Summer of Code on the worker. You can find his work here: [GitHub - lazywei/ruby-bench-docker: Docker image for ruby-bench](https://github.com/lazywei/ruby-bench-docker) Once this worker is ready to be used, I hope we can move it to the ruby-bench organization (@sam is the only one with access to the organization though), so we can go to the next step of hosting it and making it run. ---
**Author:** ![lazywei](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/lazywei/32/964_2.png) [@lazywei](https://community.miniprofiler.com/u/lazywei)\ **Post date:** [August 19, 2014, 9:23am UTC](https://community.miniprofiler.com/t/runner-progress/208/12 "2014-08-19T09:23:39Z")
> [@dmathieu](#): > > Once this worker is ready to be used, I hope we can move it to the ruby-bench organization (@sam is the only one with access to the organization though), so we can go to the next step of hosting it and making it run. I think we can start trying to run some simple benchmarks. At the same time, I’d love to transfer it to ruby-bench organization. ---
**Author:** ![wpp](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/wpp/32/811_2.png) [@wpp](https://community.miniprofiler.com/u/wpp)\ **Post date:** [August 19, 2014, 7:21pm UTC](https://community.miniprofiler.com/t/runner-progress/208/13 "2014-08-19T19:21:57Z")
My questions about the app might be best explained with this proposed UI-change: The side-navigation would be used to select one or more runners/benchmark and combinations of the two (for comparing). E.g. select Ninefold (`runner`) and Discourse Benchmark (`benchmark`). Now select 2 different `runs` to compare against e.g. `ruby-funny-falcon-patch` vs. `ruby-2.1.0-p-1` In my mind the models for this example would look something like: (diff = different the current app) - Runner - Benchmark (diff - a model as opposed to an attribute in result) - Run - Result (diff and not sure about this\*) \*Would work if we take sam’s example from above, but how would one generalize it for different benchmarks? I think we should be fine if we just get this working with the current discourse benchmark and worry about generalizing it later. before > home: > 50: 49 > 75: 50 > 90: 52 > 99: 114 after > home: > 50: 47 > 75: 48 > 90: 50 > 99: 105 ```auto runner => Ninefold benchmark => Discourse Benchmark run_before => 2.1.0-p-1 result1 => page: "home", percentile: 50, response_time: 49 result2 => page: "home", percentile: 75, response_time: 50 run_after => 2.1.0-p-1-funny-falcon_patch result1 => page: "home", percentile: 50, response_time: 47 result2 => page: "home", percentile: 75, response_time: 48 ``` Could you explain how one would model this use-case in the current application? I hope I’m not making a terrible fool out of myself, I’m sure you thought about this stuff. I’m sorry if this is super confusing I’m really tired and shouldn’t be typing… * * * \*\*Edit: Perhaps a better approach to handling the results (which might look different depending on what benchmark they came from) would be to store them directly in a column (as json) in the `runs` table? Here is an image ---
**Author:** ![sam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/sam/32/377_2.png) [@sam](https://community.miniprofiler.com/u/sam)\ **Post date:** [January 6, 2015, 11:39pm UTC](https://community.miniprofiler.com/t/runner-progress/208/14 "2015-01-06T23:39:25Z")
---
**Author:** ![sam](https://sea2.discourse-cdn.com/flex016/user_avatar/community.miniprofiler.com/sam/32/377_2.png) [@sam](https://community.miniprofiler.com/u/sam)\ **Post date:** [January 6, 2015, 11:40pm UTC](https://community.miniprofiler.com/t/runner-progress/208/15 "2015-01-06T23:40:04Z")
@wpp @lazywei @dmathieu @tgxworld @richard_ludvigh moving all rubybench discussion to: [http://community.rubybench.org/](http://community.rubybench.org/)