IncogNET 512MB $25/year
Due to it being a Tor exit, Cloudflare shows a challenge for yabs.sh redirect, so I have to use the full script URI.
For the same reason, yabsdb.com submission fails.
Going to do this a bit differently (and I've already posted the original YABS anyway).
I thought it'd be interesting to see how much of an impact virtualisation had on a YABS, so I made a VM using all 16 threads. The first observation was that when I had 16 vCores, GB5 terminated early on the multi-core Horizon Detection test. I tried a few things, thinking it was the CPU host-passthrough that was the problem, but eventually discovered it was fine running with fewer cores. Even 15 was fine, as long as there was one free vCore, but as soon as I used 16, it failed.
What's interesting about this is that the small 4k IO performance is awful in the VM, about 1/4 speed. But, what I didn't expect is that the blocks that are page size and up are somewhat quicker on the VM. I'm not sure why this is, but maybe the readbacks are more likely to end up in RAM cache.
I'd like to do a follow-up experiment sometime where the drive is a raw LVM partition as that's how my KS-LE-1 is set up for VM. This time I decided to just mount an LVM partition on /var/lib/libvirt/images. I'm still undecided whether I prefer lvextend or qemu-img resize to extend a VM's drive.
Networking
Performance is pretty much the same on host and VM. Here's the host, with the IPv6 results removed as I didn't pass them through to the VM:
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 929 Mbits/sec | 929 Mbits/sec | 19.2 ms
Scaleway | Paris, FR (10G) | 931 Mbits/sec | 935 Mbits/sec | 16.4 ms
NovoServe | North Holland, NL (40G) | busy | 936 Mbits/sec | 11.9 ms
Uztelecom | Tashkent, UZ (10G) | 873 Mbits/sec | 4.88 Mbits/sec | 87.0 ms
Clouvider | NYC, NY, US (10G) | 715 Mbits/sec | 389 Mbits/sec | 83.4 ms
Clouvider | Dallas, TX, US (10G) | 860 Mbits/sec | 312 Mbits/sec | 125 ms
Clouvider | Los Angeles, CA, US (10G) | 830 Mbits/sec | 268 Mbits/sec | 145 ms
VM:
iperf3 Network Speed Tests (IPv4):
---------------------------------
Provider | Location (Link) | Send Speed | Recv Speed | Ping
----- | ----- | ---- | ---- | ----
Clouvider | London, UK (10G) | 941 Mbits/sec | 935 Mbits/sec | 20.3 ms
Scaleway | Paris, FR (10G) | 934 Mbits/sec | 934 Mbits/sec | 16.6 ms
NovoServe | North Holland, NL (40G) | 938 Mbits/sec | 936 Mbits/sec | 11.1 ms
Uztelecom | Tashkent, UZ (10G) | 802 Mbits/sec | 290 Mbits/sec | 191 ms
Clouvider | NYC, NY, US (10G) | 894 Mbits/sec | 411 Mbits/sec | 84.0 ms
Clouvider | Dallas, TX, US (10G) | 865 Mbits/sec | 401 Mbits/sec | 122 ms
Clouvider | Los Angeles, CA, US (10G) | 832 Mbits/sec | 269 Mbits/sec | 149 ms
Geekbench
Some loss to be expected as I'm using fewer vCores, but it's still not too bad. Slightly better than 15/16 of GB5 score.
Host first:
Geekbench 5 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1441
Multi Core | 7982
Full Test | https://browser.geekbench.com/v5/cpu/18922424
and now the VM:
Geekbench 5 Benchmark Test:
---------------------------------
Test | Value
|
Single Core | 1436
Multi Core | 7633
Full Test | https://browser.geekbench.com/v5/cpu/18967843
Final notes
I'm not intending to use this VM for anything in the real-word, but I thought it was an interesting experiment.
I added this to my linux command line: iommu=pt intel_iommu=on isolcpus=1,2,3,4,5,6,7,9,10,11,12,13,14,15.
In the VM XML file, I also added this because I hate the generic CPU ID names: <cpu mode='host-passthrough' check='none' migratable='on'> <feature policy='disable' name='pdcm'/> <feature policy='disable' name='mpx'/> </cpu>
I plan to leave CPU 0/8 for the host, wireguard and routing.
I'll use 7/15 for haproxy.
I'll use 1-5/9-13 as a general pool for any ancillary VMs, like mail spool, package builder, etc.
My main web app will use 6/14 dedicated to it as its CPU 0,1 but also use that pool.
Depending on work load, I might change that balance a bit and dedicate more to the web app, or hopefully reduce the number of cores I share with the pool when I discover what the sweet spot is. I'm currently using 4 threads for the app, but there are a number of other helpers running to sync the database elsewhere too.
I scratched my head for a long time looking at the commands in the Github repository. These commands: "curl -sL yabs.sh | bash" or "wget -qO- yabs.sh | bash"
What had me confused was, "How do wget and curl know where to find the yabs.sh shell script file?"
It took a little head-scratching, but I finally figured out that they used a trick. It turns out that "yabs.sh" serves two purposes. It is the filename of the shell script. It is also a URL. The trick is that the URL (yabs.sh) redirects to the Github page so you don't notice it. (In case anyone cares, ".sh" is the country code for St. Helena, a British island in the South Atlantic.)
So ... https://yabs.sh redirects to the Github page. That's the trick. The "-s" and "-q" in the commands stand for "silent mode" and "quiet mode" respectively. I speculate that they were included to hide the trick, which shows up in the more verbose output. Note that I omitted the "-q" in the command below, to expose the more verbose output:
After that, there is a download progress bar, and then the script launches and runs the benchmark tests.
Edit, later:
I know this post will get lost in the overall thread. I wondered whether anyone cares, which is why I posted it here. It seemed pointless to start a new thread for a minor triviality like this. Maybe it was clearly obvious to everyone else, other than me. Sort of a "Yeah? So? What's the point?" situation.
@chimichurri said:
Hosthatch BF2022 2TB Amsterdam, 110 dollars every 3 years
... @dosai Big thanks for the tip! Just like you said, network is fast, my man
I clearly wasn't paying enough attention to everything that was going on! I thought the Polar Bear was the best value 2TB backup option!
@ralf said: Going to do this a bit differently (and I've already posted the original YABS anyway).
Hey @ralf - just wanted to say this was a very awesome and interesting read! Thanks for taking the time to test out the host vs a host-sized VM and compare the two results! If I'm not mistaken, I believe @Not_Oles has done some lengthy experiments testing YABS VM performance vs host performance as well in the past and had similar findings.
@xleet said: It took a little head-scratching, but I finally figured out that they used a trick
Heh -- yes, it is a trick! Can't say it was my idea, though, as I "borrowed" the idea from another widely popular benchmarking script (bench.sh). Appreciate the explanation to all those who may also be wondering the same thing as you
@chimichurri said:
Hosthatch BF2022 2TB Amsterdam, 110 dollars every 3 years
... @dosai Big thanks for the tip! Just like you said, network is fast, my man
I clearly wasn't paying enough attention to everything that was going on! I thought the Polar Bear was the best value 2TB backup option!
Indeed, I'd missed noticing it too. Unfortunately now gone; maybe back at New Years?
What's interesting about this is that the small 4k IO performance is awful in the VM, about 1/4 speed. But, what I didn't expect is that the blocks that are page size and up are somewhat quicker on the VM. I'm not sure why this is, but maybe the readbacks are more likely to end up in RAM cache.
depending on how you deploy your VMs and especially if you're using qcow2 images, it might be worth it to play around with preallocation and the cluster_size. I usually set the latter to something like 512k or 1M which can help quite a bit with IO.
though this all is much more noticable on spinning rust setups compared to ssd/nvme nowadays.
What jumped out at me was the tiny 5 point difference between the single core Geekbench 5 scores, 1441 for the Host and 1436 for the VM. I would have expected the difference to be between 50 and 150. How do you have such a low virtualization penalty?
@Mason said: Hey @ralf - just wanted to say this was a very awesome and interesting read! Thanks for taking the time to test out the host vs a host-sized VM and compare the two results! If I'm not mistaken, I believe @Not_Oles has done some lengthy experiments testing YABS VM performance vs host performance as well in the past and had similar findings.
@Not_Oles said: What jumped out at me was the tiny 5 point difference between the single core Geekbench 5 scores, 1441 for the Host and 1436 for the VM. I would have expected the difference to be between 50 and 150. How do you have such a low virtualization penalty?
I'll try another yabs run again when I get some free time, but I wonder if you'd lost performance because you'd not added the necessary things to the grub command line.
In /etc/default/grub find GRUB_CMDLINE_LINUX and add iommu=pt intel_iommu=on
WARNING: I think this should be iommu=pt amd_iommu=on for AMD, but I've not tried it.
Oh, and the other thing... I've used isolcpus=1,2,....,7,9,10,...,15 so that linux can only schedule on cores 0 and 8 (the same physical core). Then in my vcpupin, I explicitly bound vcpu 0 to 1, vcpu 1 to 2, etc... This means that the core will be completely idle except for the VM process. If you don't do that, it could up end up competing with all the normal kernel things on the host. Remember that the linux scheduler will always try to put tasks on the lowest numbered cores it can.
@Not_Oles said: What jumped out at me was the tiny 5 point difference between the single core Geekbench 5 scores, 1441 for the Host and 1436 for the VM. I would have expected the difference to be between 50 and 150. How do you have such a low virtualization penalty?
I'll try another yabs run again when I get some free time, but I wonder if you'd lost performance because you'd not added the necessary things to the grub command line.
In /etc/default/grub find GRUB_CMDLINE_LINUX and add iommu=pt intel_iommu=on
WARNING: I think this should be iommu=pt amd_iommu=on for AMD, but I've not tried it.
Oh, and the other thing... I've used isolcpus=1,2,....,7,9,10,...,15 so that linux can only schedule on cores 0 and 8 (the same physical core). Then in my vcpupin, I explicitly bound vcpu 0 to 1, vcpu 1 to 2, etc... This means that the core will be completely idle except for the VM process. If you don't do that, it could up end up competing with all the normal kernel things on the host. Remember that the linux scheduler will always try to put tasks on the lowest numbered cores it can.
Thanks for your 🌟 suggestions! 🌟 I will study up, try some stuff, and post something. Thanks again!
@ralf said: In /etc/default/grub find GRUB_CMDLINE_LINUX and add iommu=pt intel_iommu=on
WARNING: I think this should be iommu=pt amd_iommu=on for AMD, but I've not tried it.
Oh, and the other thing... I've used isolcpus=1,2,....,7,9,10,...,15 so that linux can only schedule on cores 0 and 8 (the same physical core). Then in my vcpupin, I explicitly bound vcpu 0 to 1, vcpu 1 to 2, etc... This means that the core will be completely idle except for the VM process. If you don't do that, it could up end up competing with all the normal kernel things on the host. Remember that the linux scheduler will always try to put tasks on the lowest numbered cores it can.
Webhosting24 BlackFriday 2021 Special KVM NVMe VPS, upgraded from 10x10x10, in Munich, $13.52/year.
Compared to previous benchmark from 2021-April, both CPU score and I/O score decreased.
Comments
Liga Hosting CM deal Amsterdam €4.99/m
Amsterdam send/receive at 12pm
Liga Hosting CM Deal Seattle €1.69/m
HostHatch 10TB Storage VM 2022 NL (2 years recurring)
Jaysus! Blistering speeds.
LigaHosting Seattle CM sale.
ServerStatus , slackvpn , linux mirrors
Recommended Providers: Servarica(aff) | HostBRR(aff) | SolidVPS(aff) | RackNerd(aff) | KuroIT(aff) | AlphaVPS(aff) | HostDare(aff)
IncogNET 512MB $25/year
Due to it being a Tor exit, Cloudflare shows a challenge for
yabs.shredirect, so I have to use the full script URI.For the same reason,
yabsdb.comsubmission fails.MicroNode by NatVPS Instance in Korea
EvolutionHost Developer SSD, normally €10/month
iHostART KVM 1GB, normally €5.40/year
iHostART OpenVZ 50GB HDD, €5.50/year
IPv6 is via WireGuard tunnel to iHostART KVM 1GB.
We accept Karma donations for the last flan. 🍮 affbrr
Advin Servers
KVM Standard S Black Friday (2 CPU 16GB RAM)
Los Angeles
@Mason you probably never expected this to happen... but looks it can happen and you need one more if in code, lmao.
Haven't bought a single service in VirMach Great Ryzen 2022 - 2023 Flash Sale.
Got to keep the templates small when we sell packages with 3Gb disk!
Yeah.... in retrospect, I should have known this issue would arise when I accepted the PR last week. Will get it patched up soon(tm)
yabs daddy
the LXC templates even come without syslog-ng.
Free NAT KVM | Free NAT LXC
Going to do this a bit differently (and I've already posted the original YABS anyway).
I thought it'd be interesting to see how much of an impact virtualisation had on a YABS, so I made a VM using all 16 threads. The first observation was that when I had 16 vCores, GB5 terminated early on the multi-core Horizon Detection test. I tried a few things, thinking it was the CPU host-passthrough that was the problem, but eventually discovered it was fine running with fewer cores. Even 15 was fine, as long as there was one free vCore, but as soon as I used 16, it failed.
Summary
First up host top-level summary:
and the VM:
FIO
The next section is what really surprised me. First up the host, which is using an ext3 FS on an LVM 2-disk mirror:
And now the VM, which is running on a ext3 FS, on a qcow2 file mounted on an ext3 FS on an LVM 2-disk mirror:
What's interesting about this is that the small 4k IO performance is awful in the VM, about 1/4 speed. But, what I didn't expect is that the blocks that are page size and up are somewhat quicker on the VM. I'm not sure why this is, but maybe the readbacks are more likely to end up in RAM cache.
I'd like to do a follow-up experiment sometime where the drive is a raw LVM partition as that's how my KS-LE-1 is set up for VM. This time I decided to just mount an LVM partition on
/var/lib/libvirt/images. I'm still undecided whether I preferlvextendorqemu-img resizeto extend a VM's drive.Networking
Performance is pretty much the same on host and VM. Here's the host, with the IPv6 results removed as I didn't pass them through to the VM:
VM:
Geekbench
Some loss to be expected as I'm using fewer vCores, but it's still not too bad. Slightly better than 15/16 of GB5 score.
Host first:
and now the VM:
Final notes
I'm not intending to use this VM for anything in the real-word, but I thought it was an interesting experiment.
I added this to my linux command line:
iommu=pt intel_iommu=on isolcpus=1,2,3,4,5,6,7,9,10,11,12,13,14,15.In the VM XML file, I also added this because I hate the generic CPU ID names:
<cpu mode='host-passthrough' check='none' migratable='on'><feature policy='disable' name='pdcm'/><feature policy='disable' name='mpx'/></cpu>I plan to leave CPU 0/8 for the host, wireguard and routing.
I'll use 7/15 for haproxy.
I'll use 1-5/9-13 as a general pool for any ancillary VMs, like mail spool, package builder, etc.
My main web app will use 6/14 dedicated to it as its CPU 0,1 but also use that pool.
Depending on work load, I might change that balance a bit and dedicate more to the web app, or hopefully reduce the number of cores I share with the pool when I discover what the sweet spot is. I'm currently using 4 threads for the app, but there are a number of other helpers running to sync the database elsewhere too.
Fascinating to see it broken down that way and also you info on how you plan to use the cores. Thanks for sharing!
LigaHosting KVM SSD - US CB - LES CB
The Yeti has left the building.
Monster @ $100 Year
HostCram LLC - Web Hosting Built For Speed, Reliability, Security & Uptime! [We operate AS39618]
Hosthatch BF2022 2TB Amsterdam, 110 dollars every 3 years
@dosai Big thanks for the tip! Just like you said, network is fast, my man
Contribute your idling VPS/dedi (link), Android (link) or iOS (link) devices to medical research
I was trying out YABS for the first time. Here is where you find the shell script, more about that later:
https://github.com/masonr/yet-another-bench-script
I scratched my head for a long time looking at the commands in the Github repository. These commands: "curl -sL yabs.sh | bash" or "wget -qO- yabs.sh | bash"
What had me confused was, "How do wget and curl know where to find the yabs.sh shell script file?"
It took a little head-scratching, but I finally figured out that they used a trick. It turns out that "yabs.sh" serves two purposes. It is the filename of the shell script. It is also a URL. The trick is that the URL (yabs.sh) redirects to the Github page so you don't notice it. (In case anyone cares, ".sh" is the country code for St. Helena, a British island in the South Atlantic.)
So ... https://yabs.sh redirects to the Github page. That's the trick. The "-s" and "-q" in the commands stand for "silent mode" and "quiet mode" respectively. I speculate that they were included to hide the trick, which shows up in the more verbose output. Note that I omitted the "-q" in the command below, to expose the more verbose output:
After that, there is a download progress bar, and then the script launches and runs the benchmark tests.
Edit, later:
I know this post will get lost in the overall thread. I wondered whether anyone cares, which is why I posted it here. It seemed pointless to start a new thread for a minor triviality like this. Maybe it was clearly obvious to everyone else, other than me. Sort of a "Yeah? So? What's the point?" situation.
I clearly wasn't paying enough attention to everything that was going on! I thought the Polar Bear was the best value 2TB backup option!
Hey @ralf - just wanted to say this was a very awesome and interesting read! Thanks for taking the time to test out the host vs a host-sized VM and compare the two results! If I'm not mistaken, I believe @Not_Oles has done some lengthy experiments testing YABS VM performance vs host performance as well in the past and had similar findings.
yabs daddy
Heh -- yes, it is a trick! Can't say it was my idea, though, as I "borrowed" the idea from another widely popular benchmarking script (bench.sh). Appreciate the explanation to all those who may also be wondering the same thing as you
yabs daddy
Indeed, I'd missed noticing it too. Unfortunately now gone; maybe back at New Years?
I have an older deal
$84/y
2 CPU core
2.5 TB storage
2.5 GB RAM
6 TB bandwidth
This new deal would have been saving me nearly $50 a year 👀🤦
1 CPU core
2 TB storage
1 GB RAM
10 TB bandwidth
$50 per year
$110 per 3 years (doubled RAM and bandwidth for three year payments)
@ralf @tarasis You may still be able to get it (quote from OGF HH BF thread)
Contribute your idling VPS/dedi (link), Android (link) or iOS (link) devices to medical research
nice testing and writeup!
depending on how you deploy your VMs and especially if you're using qcow2 images, it might be worth it to play around with preallocation and the cluster_size. I usually set the latter to something like 512k or 1M which can help quite a bit with IO.
though this all is much more noticable on spinning rust setups compared to ssd/nvme nowadays.
AWS lighstail, 5$ p.m., last benchmark before final shutdown after rockstable 3+ years of uptime
>Geekbench 5 Benchmark Test:
Test | Value
|
Single Core | 1441
Multi Core | 7982
Full Test | https://browser.geekbench.com/v5/cpu/18922424
and now the VM:
>Geekbench 5 Benchmark Test:
Test | Value
|
Single Core | 1436
Multi Core | 7633
Full Test | https://browser.geekbench.com/v5/cpu/18967843
Hi @ralf!
Thanks for posting this. Very interesting!
What jumped out at me was the tiny 5 point difference between the single core Geekbench 5 scores, 1441 for the Host and 1436 for the VM. I would have expected the difference to be between 50 and 150. How do you have such a low virtualization penalty?
Hi @Mason!
Thanks for your kind words! In case anybody is interested, here are links to previous posts about the difference between host and VM performance:
When running qemu on Hetzner AX101, how much difference does -cpu host and -enable-kvm make?
results with qemu's default processor (as in the first test, above) but now flying the -enable-kvm flag
Here is a link to your previous comment:
This is why I always request for CPU passthrough
Best wishes from the desert! 🏜️
I'll try another yabs run again when I get some free time, but I wonder if you'd lost performance because you'd not added the necessary things to the grub command line.
In
/etc/default/grubfindGRUB_CMDLINE_LINUXand addiommu=pt intel_iommu=onWARNING: I think this should be
iommu=pt amd_iommu=onfor AMD, but I've not tried it.Oh, and the other thing... I've used
isolcpus=1,2,....,7,9,10,...,15so that linux can only schedule on cores 0 and 8 (the same physical core). Then in my vcpupin, I explicitly bound vcpu 0 to 1, vcpu 1 to 2, etc... This means that the core will be completely idle except for the VM process. If you don't do that, it could up end up competing with all the normal kernel things on the host. Remember that the linux scheduler will always try to put tasks on the lowest numbered cores it can.Thanks for your 🌟 suggestions! 🌟 I will study up, try some stuff, and post something. Thanks again!
Amsterdam - KVM Memory VPS - KVM Beta High
Prior to transfer to @iaecm - https://lowendspirit.com/discussion/4983/post-bf-buyers-remorse-tuesday-vps-giveaway
EDIT: realised that this was after a fresh re-install, so the net speeds might be slightly lower than they should as I didn't change the MTU.
Naranjatech: 1 CPU, 1GB RAM and 16GB NVMe 11,50 €/year
MaxKVM ADX-2 Singapore: Monthly: $2.90
1x vCPU (EPYC 7702p, dedicated)
2GB RAM (ECC DDR4)
60GB SSD (PCIe4 NVMe SSD)
2TB Transfer (Inbound + Outbound, monthly)
10Gbps+ Port Speed (Dual 40G uplink)
https://dttech.top - Personal Site.
Server Factory - Ryzen S (2 dedicated vCores)
Prior to transfer to @contactwajeeh - https://lowendspirit.com/discussion/4983/post-bf-buyers-remorse-tuesday-vps-giveaway
I'm a bit sad to let this one go, it's been great!
HostCram KVM-3C - $21 Quarterly (same as BF, but slightly higher price and bigger drive)
Prior to transfer to @webkesh - https://lowendspirit.com/discussion/4983/post-bf-buyers-remorse-tuesday-vps-giveaway
Another fine VPS, amazing performance for the price.
Old HH year end deal 2 vCPU, 2GB Ram, 2.5TB disk, 6TB BW $84/y
New HH BF 2TB, $110/3y. 1vCPU, 2GB Ram, 2TB disk, 20TB
Much better net connection, slower disk (although could be hit with others testing), and slower core but not by a lot. $84/y vs $36/y
Psychz Network (Copy from MJJ-loc)
Notice the Disk IO
Ontario Dildo Inspector
HostHatch Sweden
Hi @ralf!
The iommu/VT-d already might be enabled on Darkstar because I can see:
I have to look more at the scheduling and run more tests.
Thanks again for your helpful comments!
Tom
My new firewall - will probably run VMs on it too given that its way overspec'd for pure firewall duty (16gb / fast 1tb nvme)
Spaceberg VDS Nasty - 1 (Stack)
Our Sweety - 1 package. HDD, NVMe cached
Spaceberg.cc - Your favorite Seedbox provider!
greencloudvps
SG Storage
after migration to new DC
I bench YABS 24/7/365 unless it's a leap year.
spaceberg
france
I bench YABS 24/7/365 unless it's a leap year.
greencloudvps
San Jose
$40/3 years (double cpu during pre order)
I bench YABS 24/7/365 unless it's a leap year.
Webhosting24 BlackFriday 2021 Special KVM NVMe VPS, upgraded from 10x10x10, in Munich, $13.52/year.
Compared to previous benchmark from 2021-April, both CPU score and I/O score decreased.
We accept Karma donations for the last flan. 🍮 affbrr
holy shit, what a beast
whats wrong with formating. does not work for me!
``
woot@localhost:~# curl -sL yabs.sh | bash
## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ##
Yet-Another-Bench-Script
v2022-12-04
https://github.com/masonr/yet-another-bench-script
## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ## ##
Thu 08 Dec 2022 06:13:15 AM PST
Basic System Information:
Uptime : 17 days, 5 hours, 15 minutes
Processor : AMD Ryzen 9 5900X 12-Core Processor
CPU cores : 1 @ 3693.060 MHz
AES-NI : ✔ Enabled
VM-x/AMD-V : ✔ Enabled
RAM : 976.4 MiB
Swap : 0.0 KiB
Disk : 270.5 GiB
Distro : Debian GNU/Linux 11 (bullseye)
Kernel : 5.10.0-11-amd64
fio Disk Speed Tests (Mixed R/W 50/50):
iperf3 Network Speed Tests (IPv4):
Geekbench 5 Benchmark Test:
Test | Value
|
Single Core | 1639
Multi Core | 1620
``
Wow
What provider?
add one more ` .. i do it with 3 ```
ServerStatus , slackvpn , linux mirrors
Recommended Providers: Servarica(aff) | HostBRR(aff) | SolidVPS(aff) | RackNerd(aff) | KuroIT(aff) | AlphaVPS(aff) | HostDare(aff)
Another Post-CM InceptionHosting buy.. this time in Netherlands. Great little box
ServerStatus , slackvpn , linux mirrors
Recommended Providers: Servarica(aff) | HostBRR(aff) | SolidVPS(aff) | RackNerd(aff) | KuroIT(aff) | AlphaVPS(aff) | HostDare(aff)
holy shit, what a beast
Nice. Which provider? Plan?
buyvm
Premium