I'll respond tomorrow when I'm back at my computer. But my gut instinct is that your chronograph results are accurate.
Thanks! I really appreciate the feedback and guidance.
From thermocouples to blood pressure cuffs to blood sugar testers, I know from personal experience and reading spec sheets that getting inaccurate and/or inconsistent results in to be expected from a consumer product and that's what your chronograph appears to be, especially given its price. I know growing up, I wondered why lab equipment was so expensive compared to what you can get from the average store. Turns out it was the equipment's level of accuracy and precision. That's a major reason why professional or mission critical equipment costs significantly more than the "everyday" version of the equipment.
As for whether or not your chronograph is inaccurate, I can't say, but it wouldn't surprise me one bit if it's reading lower readings than it should. I guess the real questions are whether you can re-calibrate it and if the inaccuracy is consistent, i.e. you're still able to maintain precision.
Agree - back when I worked in a chemistry lab we spent a lot of time (and money) on purchasing reference materials and baselining our equipment to profile our accuracy against known references. I was trying to think of a way to apply that to blasters, e.g. find a fairly consistent platform and potentially modify it with a fairly consistent spring, then use that as a reference platform. If my readings are "x" fps lower than the reference then we know where my setup stands. I can also use that approach to monitor for drift over time or under various conditions.
It's hard to imagine getting precise numbers at these low fps ranges when the manufacturer is marketing to the firearms market where the fps ranges are orders of magnitude higher. I thought I might have a chance with this one because they also market to archers, but you never know until you give it a go. I don't think these cheapo / entry-level devices have a field calibration method, or at least one that I have found. I'll hit up the manufacturer again with another round of questions.
I'm not an expert on chronographs (I don't even own one), but normally people measure their shots 6inches away; not your 12in. Coop, Drac, and Orangemodworks chrono at 6 inches.
Sorry about overlooking that detail. I lumped all my muzzle readings into a category of "under 1 foot" because of the different designs of the blasters. I have readings from my two modified Retaliators that are using aftermarket stefan kits from Artifact and Worker. Their barrels (about 4") are nested inside the outer barrel of the pump grip on my Retaliators, so technically the muzzle is 6" further back from the front of the blaster. I don't know how significant this is, and I didn't feel like switching back to the stock top prime with some sort of bolt grip. That 10kg Aussie spring is tough to prime haha. Anyhow I just label them as "1 ft" when capturing the data on my phone and it carried forward to my laptop when I was plotting.
I'm still not sure of the best way to characterize this measurement. Advice here is appreciated!
Darts also play a significant role in chrono readings. An elite may mass more than a koosh but less than a waffle, for example. Which is ALSO something you could (& should) test - especially the accuracy between types, as this is a fairly under supported area in nerf research.
IMO, your actual results don't matter as long as YOUR setup is consistent- if you go on to present (beautiful !) statistics like these for many categories of blaster and mod, the actual result can be manipulated later if your chrono is found to be ~10% slow (or whatever). It is only an issue if you are changing the setup between blasters (different lighting, angle, w/e) and/or your results are varying inconsistently ( between shots or blasters).
Speaking of, how is your setup? That's an important part of statistics taking too. Do you fire from a bench, aim each one, ...? Do you have good, consistent lighting for the chrono? Are your darts fresh elites and all one color? Etc. etc.
I would absolutely love to see more, especially a set of stock/modded Stryfes. Then look into the SCAR barrel - I'd love to see results on that. The Hammershot would also probably warrant an extensive test (stock, spacered, 3d printed 7-cylinder, new spring). Obviously the more blasters with the more darts and setups, the better, but to keep a reasonable lid on it I'd leave it with those four to start (Retal, Caliburn, Stryfe, HS).
Thanks for the comment on "(beautiful !) statistics"! My day job focuses on condensing millions to billions of data points into manageable visualizations, so I enjoy applying that to my hobbies (I'm also into dart frogs where I have a large source of temperature and humidity data). The nerf hobby caught my attention with fps / accuracy / precision data, and I thought this would be an interesting challenge, especially with the variety of variables you can introduce with different mods.
I forgot to mention the darts. I baselined the stock Retaliator with streamlines and ekind waffles. Here's a quick screen grab of the summary data from two runs where the variable was dart type:
This data indicated the two dart types yielded the same FPS results. I can repeat them with a different blaster, but they were pretty consistent. I look mainly at the 95th pct confidence interval of the population because that gives me a better idea of what to expect from any given shot from a blaster. This table has all the base stats I use to make those plots, the population of results is below this summary in my worksheet and I can clean all that up and share it if anyone is interested in the raw data too.
I'll admit my setup isn't super consistent yet. I'm hand firing the blasters through the Caldwell on a bar-height table. I did pickup the indoor lighting kit to improve the consistency of the readings from the chronograph. I shoot indoors to keep the temp / RH consistent (we set the target RH for the house and it's really consistent, especially in the summer). I'm in a 15 foot room so I can get my 1, 6 and 12 foot ranges. 18 and 24 are fired through a double door sized opening (best I can do). Using this setup I can extend the ranges to ~50 ft across the house if I get an accurate enough blaster to hit the chronograph from that distance. There's a sterilite bin behind the Caldwell with a towel taped to it that catches darts. I've been thinking of some kind of cart top mount for the worker p-mag / hex-mags that may get a consistent firing angle, but I thought I would get a set of range tests manually before taking that step. If anyone has additional ideas I'm open to feedback!
In terms of darts, most of these tests are focused on stefans / short darts. I've tested worker, artifact FVJ's, and ACC Gen 2 soft and hard tips. I mostly test with the hard tips now. I have not tried to make my own [yet]. I have batches of 100 that I'll cycle through for a test, loaded in a variety of worker stefan mags that I have on hand. I have not tried stock darts in the Caliburn yet.
One of the tests I mentioned above, reference testing, is something I need to circle back to. One of the concepts I've used in the lab was to periodically retest a configuration and compare the results that I got last week, 2 weeks ago, etc to observe the consistency of the setup over time. In the chem lab we were monitoring for "machine drift" but here it's more likely operator drift or a busted blaster (e.g. I had an o-ring work itself loose in one of the Artifact stefan tests and you could see the impact in the course of the 100 shot test through a step down in the fps results). I'll get back to re-testing a configuration with one of the modded Retaliators (my original "stock Ret" is now torn apart / dremeled out for an expanded plunger tube mod).
I agree I'm after nailing down the consistency of my setup, and I can test for that through a series of reproducibility tests. That said, given how many blasters are out there, I'd also like to see how we can share results in the community and try to account for variability from setup to setup through some kind of standardization process. Maybe I'm complicating this too much, but that idea of coming up with a blaster that is widely available and consistent, or consistent with some specific mods, could help us all adjust our sharing methodologies. Don't get me wrong, I'd love to test a bunch of different blasters, but I think a deeper dive into coming up with a collection methodology and standardized analysis would let us all profile a lot more blasters. It would be cool to pull together an online repository of results as well (if this doesn't already exist).
I haven't gotten into flywheels [much] yet. My kiddos are young and I enjoy modding blasters they can use so I focused on springers since I'm leery about having them playing with lipos right now. We do have a couple of stryfe's that I'll get to eventually, but I kind of got hooked on springers. And man is the Caliburn fun lol!
I too am intrigued by the SCAR barrel. I have been messing with a design that would slip on to the Caliburn. I am starting without a twist. I've read the improvement in accuracy may be from a stabilizing effect rather than the twist so I added 4 parallel ridges to this design and some exhaust holes (it looked cool but maybe they are not needed - I can remove them in another design). I thought I would iterate on this for a while then add a twist later.
Anyhow this idea was the main driver behind coming up with a way to baseline accuracy as a function of distance. Here's what I started with.
Front orientation (you can see the teeth):
Rear orientation that slips on the barrel:
My first print was too narrow for the OD of the Caliburn barrel so I need to tune the design a bit more. I also wanted everyone's feedback on this thread before I did a distance profiling on the Caliburn since that work is pretty time consuming (500 shots or more!). I need to do more reading on the SCAR barrel and design a set of experiments around different attachment designs that covers the key variables that surfaced on many of the forum discussions. Feedback here is also more than welcome. Many of these experiments will be long running.
I think your data looks pretty good in comparison to some other spring data I've seen. I would only really be able to tell you about flywheel data, because that's my specialization. Compliments to the way you layed it out too.
Thanks I'm glad the feedback so far seems to be the numbers are coming in OK. My main concern was the delta I was seeing with the reported numbers from folks like Coop and Drac on stock/modded blasters, and Captain Slug's Calibrun numbers. I seem consistently lower, which is what got me thinking on calibration, cross-comparison, and possibly reference configurations. And thanks for the compliment on layout!
Apologies for the long post! I squirreled away a lot of data over the past few months that I'm trying to organize. It took a few months to compile nearly two thousand shots of data behind these plots and others I haven't published yet. I figured it was time to gather my thoughts and some feedback before really getting into deep profiling of specific configurations.
I really appreciate all the feedback and help! --Jim