v2.0.0 #15
juniper
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "experimental"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
A huge update, rewriting huge chunks of the code in C.
cccrate, the C part of the API should be its own standalone library that can be installed systemwide (Perhaps a toggle between the two?)Rewrite compute function in C or C++ for 16 bit sigmoid networks(Postponed for later updates)astra-gpufunctioncompute_gpu_sigf32's output with something C-compatiblev1.3.0to v2.0.0Given the scale of this next Astra API update, I've changed the version number from v1.3.0 to v2.0.0 - It's just too big to be a major update on its own.
My hope for it is to replace all compute-related Rust code with C, then call that code from Rust.
This would also make the API work better for non-rust languages; the Rust stuff is just a wrapper around a C API in this situation, really - With the exception of the GPU code, since I don't hate myself enough to rewrite that in C or C++
I wouldn't expect the first v2.0.0 release of the API to be substantially better or particularly different if you're using the Rust API, but this does open up a bunch of optimization opportunities that may be difficult otherwise - My main focus is memory usage.
Unlike normal, I might ship several pre-releases once it's in a semi-ready state, with some features missing (Likely things like 16-bit support which aren't as much of a priority compared to 32-bit)
v2.0.0to WIP: v2.0.0I'm mostly happy with the C code, now. Currently AVX512 is still unsupported and AVX2 is forcibly required - So this will have to change.
I would expect all modern processors to have AVX2 at least, however for non-x86 devices I see how this could be problematic - ARM, RISC-V, etc have no AVX.
I may do this similarly to how I have split the library already - there's
libastra32.sofor 32-bit support andlibastra16.sofor 16-bit support (although unfinished as of writing)Though, I worry this will split the library up too much - You'd have
libastra32-avx2.so,libastra32-avx512.soand their 16-bit counterparts added on with the other two files.As of right now, the rewrite is functional with some slight performance regressions - Including AVX2, AVX512 and non-AVX implementations, but currently missing 16-bit floating point support.
Comparing two of the versions, on the v2.0.0 experimental branch, I'm getting these benchmark results on an i5-10400, 64GB of DDR4-3200 and no GPU (using the iGPU)
57.8ms (No AVX
57.4ms (AVX2)
AVX512 isn't supported on 10th gen intel, so it wasn't run.
3.78ms (GPU, Intel UHD Graphics 630)
As opposed to ~41ms on the same hardware on the CPU on v1.1.0
It runs well, but not as well as I'd like.
After more research, it seems a similar performance regression is in v1.2.2 - So this is acceptable to me. I will, however, continue to work towards beating the original v1.1.0 benchmarks.
Everything is ready for release - Some more testing is required but I am confident in this version being improvement over the previous release.
Ready for release, just needs some minor tweaks before pushing it.
One thing to add: I've discovered a way the performance could be improved (by up to about 12ms in compute on my system in the benchmarks, which would compound greatly in training).
This won't be packaged in v2.0.0 though, I think it'll be in v2.0.1
WIP: v2.0.0to v2.0.0