Several architectures for large-scale IoT edge container cluster management - 2-HashiCorp solution Nomad

Review above

1. Introduction to several architectures for large-scale IoT edge container cluster management - 0-edge containers and architectures
2. Several architectures for large-scale IoT edge container cluster management - 1-Rancher+K3s

📚️Reference:
IoT Edge Computing series

HashiCorp Solutions - Nomad + Docker

Brief introduction

Nomad: A simple and flexible scheduler and orchestrator that deploys and manages container and non-containerized applications at scale on-premises and in the cloud

Nomad enables developers to deploy applications using declarative infrastructure as code. Nomad uses bin packing to schedule work efficiently and optimize resource utilization.

Nomad with itsSimplesexflexibility、ScalabilityandPerformanceDistinguish from related tools. Nomad’s synergies and integration points with HashiCorp Terraform, Consul, and Vault make it uniquely suited for easy integration into an organization’s existing workflows, minimizing time-to-market for critical initiatives.

Use Nomad to schedule edge workloads closer to your users. Connect edge services with Nomad’s on-premises service discovery. Seamlessly handles unstable Nomad client node connections.

Reference architecture

When adopting edge computing/containers, there are things like managing heterogeneous devices (different processors, operating systems, etc.), resource-constrained devices, and intermittent connectivity.

Nomad addresses these challenges, making it an attractive edge coordinator. The Nomad client agent is a single binary file with a small footprint, limited resource consumption, and the ability to run on different types of devices. In addition, Nomad supports geographically distant clients, which means that Nomad server clusters do not need to run close to clients.

With Nomad 1.3, native service discovery simplifies the process of connecting Nomad tasks, where you don’t need to use a single service mesh and no longer need to manage a separate Consul cluster. Nomad’s native service discovery also eliminates the need to install the Consul agent on every edge device. This further reduces Nomad’s resource footprint, so you can run and support more workloads at the edge. In addition, disconnected client assignments can gracefully reconnect, handling situations where edge devices experience network latency or temporary loss of connectivity.

As shown in the figure below,

1. An on-premises data center or cloud that will host a cluster of Nomad servers and a client for unified management
2. The edge side runs the Nomad client.

Program advantages

• Deploy containers and legacy applications: Nomad’s flexibility as a coordinator enables organizations to run containers simultaneously on the same infrastructure,Legacy and batch applications。 Nomad brings the core orchestration benefits to legacy applications without containerization.
• Simple and reliable: Nomad runs as a single binary and is completely self-contained—combining resource management and scheduling into a single system. Nomad does not require any external services for storage or orchestration. Nomad handles application, node, and driver failures automatically. Nomad is distributed and resilient, using leader election and state replication to provide high availability in the event of a failure.
• Low resource footprint: The Nomad client agent is a single binary file with a small footprint and low resource usage; Nomad version 1.3 and later native service discovery also eliminates the need to install the Consul agent on every edge device. This further reduces Nomad’s resource footprint.
• Device plug-insand GPU support: Nomad provides built-in support for GPU workloads such as machine learning (ML) and artificial intelligence (AI). Nomad uses devices Plugins to automatically detect and utilize resources from hardware devices such as GPUs, FPGAs, and TPUs. The plugins are rich and support:
  • Docker
  • Isolated Fork/Exec
  • Java
  • Podman
  • QEMU
  • Raw Fork/Exec
  • containerd
  • Nvidia
  • USB
  • …
• Proven scalability: Nomad optimistically concurrency, which increases throughput and reduces latency for workloads. Nomad has been shown to scale to clusters of 10K+ nodes in a live production environment.
• HashiCorpEcosystem: Nomad integrates seamlessly with Terraform, Consul, Vault for configuration, service discovery, and secret management. Meet more complex edge container management needs.

Scheme disadvantages

• Not compatible with the Kubernetes ecosystem: Nomad is a completely different container orchestration/scheduling solution from Kubernetes, so it is not compatible with the Kubernetes ecosystem. There are associated learning costs and additional labor costs.
• Fewer management features: It is also because it is not compatible with the Kubernetes ecosystem, so it is completely unable to enjoy the functional dividends brought by the huge Kubernetes ecosystem. There are relatively few management functions. If additional automation capabilities are required, Terraform integration is required; Additional service discovery capabilities are required, and Consul integration is required; Additional secret management capabilities are required, and Vault integration is required.
• The UI is simple

Keep reading

1. Several architectures for large-scale IoT edge container cluster management - 3-Portainer
2. Several architectures for large-scale IoT edge container cluster management - 4-Kubeedge
3. Several architectures for large-scale IoT edge container cluster management - 5 - Summary

Reference documentation

• Managing Applications at the Edge with HashiCorp Nomad