Description

Span<T> is a new ref struct introduced in C# 7.2 specification. It is a stack-only type that allows memory operations without allocation so, used for instance in very large arrays, it can be a significant performance improvement.

It is only applicable if your code is based from .NET Core 2.1 and .NET Standard 2.1. There are tons of technical documentation available about Span<T>, this post is just going to be focused in a practical demo to compare the performance of the slice method. Span<T> can't work inside Anync methods but you can work around this issue easily creating a non Async local method.

Detailed information can be found in the official Microsoft link: https://docs.microsoft.com/en-us/dotnet/api/system.span-1?view=net-5.0

Introduction

Weeks ago I decided to start creating an experimental home size "Big data" system based on Apache Spark. The first step for it is to create a distributed filesystem where Apache Spark will read and write eveything.

HDFS is the Hadoop distributed filesystem which provides features like: fault detection and recovery, huge datasets, hardware at data, etc... despite it is a Hadoop ecosystem piece, it works nice as the data distributed filesytem for Apache Spark.

Months ago I decided to build a Raspberry pi 4 cluster to try some distributed software projects on it. The main goal was to build it in a compact way and as cheap as possible. Planning all hardware buying is not a 5 minutes task so I thought about writing a short post.

This post tries to save you time if you're looking for something similar pointing what and where to buy all the stuff required and have your Raspberry 4 cluster ready to mount.

I bought all the pieces mainly in two places EBay and AliExpress. Obviously it took two weeks to have all these pieces together after they came from UK and China.