So yesterday I found another French provider which is both budget and not OVH/Online.net or a reseller thereof. This FirstHeberg seems to do everything on their own, which is nice for us.
For those who can't wait, here are the dedicated servers: http://firstheberg.com/dedies/
To test them out I got the cheapest server in stock: http://firstheberg.com/dedies/detail-atom-d525-hdd-2012
When ordering I ran into a little problem, even though they say on their website they accept customers from a lot of countries in Europe, China and the USA, the SMS verification system that needs to verify a phone number only sends to France, Belgium and Autria if my memory serves me well. I got around this limitation by just doing nothing on that order page and navigation to the customer panel, where the invoice was ready to be paid without further ado. Their panel is okay, it proved an rDNS, reinstalls and whatever you might need. As far as I can find they provide a default Windows Server 2008 R2 for free without bitching.
As opposed to the product page I got a single 1TB drive instead of two 500GBs, and I might have DDR2 instead of DDR3, but haven't checked that closely so I could be wrong. They offer a 100mbit line of which a mere 10mbit is guaranteed, but there is a gigabit port, on the serverbear you can find the speeds which range from, out the top of my head, 100KB/s to 45MB/s. Reading their website I think their servers are located in apartments.
I paid one month with paypal, and see what they're further up to.