🚀 I just need to improving myself in everyday and sharing my learned for everyone.
Hi reader 👋🏼, The adoption of DevOps practices
has become increasingly prevalent within development team today. Many organizations have integrated the role of a DevOps engineer into their teams or have expanded the responsibilities of existing team members to encompass DevOps methodologies
, akin to the Agile framework. In these scenarios, development team either recruit dedicated individuals to drive DevOps processes
or distribute these responsibilities across the job description of all team members. This approach acknowledges that DevOps
aims to enhance productivity by optimizing the workflows and responsibilities associated with each role. 👨🏻💻🧑💻👩🏻💻
📜 A crucial aspect of DevOps is the application of the Idempotence principle
(alternatively referred to as idempotent or idempotency) in coding practices. This concept ensures that Infrastructure
can be consistently managed and controlled through code. Fundamentally, it builds upon the previously established concept of Infrastructure as Code
, which is widely implemented using tools such as Ansible, Terraform, Puppet, and other.😺
🐈 It’s a important principle in IaC or several programming, particularly in the context of operations or functional that modify state or data. An operation is considered idempotent if it can be applied multiple times without changing the result beyond the initial application.
In programming, idempotence is desirable for several reasons:
Idempotence is particularly valuable in scenarios like:
It’s significant to note that idempotence is a property of an operation, nor necessarily of an entire system or application. In practice, developers usually strive to make critical operations idempotent to improvement reliability, consistency, and simplicity in the software operational. 🐈⬛
The provided example code written in Golang
illustrates the practical implementation of the idempotent concept within the context of a basic data management application.
A Person
struct is defined to represent individual records, with the Email
field serving as the unique identifier. Two simple functions are implemented:
Person
records into an in-memory database.Beginning with define a Person
struct
type Person struct {
Email string
Name string
Age int
}
Define a new schema of in-memory database with init()
func init() {
// Create the DB schema
schema := &memdb.DBSchema{
Tables: map[string]*memdb.TableSchema{
"person": &memdb.TableSchema{
Name: "person",
Indexes: map[string]*memdb.IndexSchema{
"id": &memdb.IndexSchema{
Name: "id",
Unique: true,
Indexer: &memdb.StringFieldIndex{Field: "Email"},
},
"age": &memdb.IndexSchema{
Name: "age",
Unique: false,
Indexer: &memdb.IntFieldIndex{Field: "Age"},
},
},
},
},
}
var err error
// Create a new data base
db, err = memdb.NewMemDB(schema)
if err != nil {
panic(err)
}
}
Create an Insert function for built transaction into database.
func InsertDataWithID(member *Person) error {
// Create a write transaction
txn := db.Txn(true)
// Insert the new person
fmt.Printf("Inserting ID: %s\n", member.Email)
if err := txn.Insert("person", member); err != nil {
return err
}
// Commit the transaction
txn.Commit()
// Create read-only transaction
txn = db.Txn(false)
defer txn.Abort()
return nil
}
func InsertMultipleData(people []*Person) error {
for _, p := range people {
if err := InsertDataWithID(p); err != nil {
return err
}
}
return nil
}
Create a retrieving all existing data from database and print out to console.
func ListAllValues() error {
txn := db.Txn(false)
// List all the people
it, err := txn.Get("person", "id")
if err != nil {
return err
}
fmt.Println("All the people:")
for obj := it.Next(); obj != nil; obj = it.Next() {
p := obj.(*Person)
fmt.Printf("[+] %s | %s | %d\n", p.Name, p.Email, p.Age)
}
return nil
}
Define main()
function to call both functions as insert and then retrieve them.
func main() {
// Insert some people
people := []*Person{
&Person{"[email protected]", "Joe", 30},
&Person{"[email protected]", "Lucy", 35},
&Person{"[email protected]", "Joey", 26},
&Person{"[email protected]", "Tariq", 21},
&Person{"[email protected]", "Dorothy", 53},
&Person{"[email protected]", "Joe", 35},
}
if err := InsertMultipleData(people); err != nil {
panic(err)
}
if err := ListAllValues(); err != nil {
panic(err)
}
}
You can see fully code in one file below. you can see my test data that have a duplicated index as 1 and 3.
// main.go
package main
import (
"fmt"
"github.com/hashicorp/go-memdb"
)
type Person struct {
Email string
Name string
Age int
}
var db *memdb.MemDB
func init() {
// Create the DB schema
schema := &memdb.DBSchema{
Tables: map[string]*memdb.TableSchema{
"person": &memdb.TableSchema{
Name: "person",
Indexes: map[string]*memdb.IndexSchema{
"id": &memdb.IndexSchema{
Name: "id",
Unique: true,
Indexer: &memdb.StringFieldIndex{Field: "Email"},
},
"age": &memdb.IndexSchema{
Name: "age",
Unique: false,
Indexer: &memdb.IntFieldIndex{Field: "Age"},
},
},
},
},
}
var err error
// Create a new data base
db, err = memdb.NewMemDB(schema)
if err != nil {
panic(err)
}
}
func InsertDataWithID(member *Person) error {
// Create a write transaction
txn := db.Txn(true)
// Insert the new person
fmt.Printf("Inserting ID: %s\n", member.Email)
if err := txn.Insert("person", member); err != nil {
return err
}
// Commit the transaction
txn.Commit()
// Create read-only transaction
txn = db.Txn(false)
defer txn.Abort()
return nil
}
func InsertMultipleData(people []*Person) error {
for _, p := range people {
if err := InsertDataWithID(p); err != nil {
return err
}
}
return nil
}
func ListAllValues() error {
txn := db.Txn(false)
// List all the people
it, err := txn.Get("person", "id")
if err != nil {
return err
}
fmt.Println("All the people:")
for obj := it.Next(); obj != nil; obj = it.Next() {
p := obj.(*Person)
fmt.Printf("[+] %s | %s | %d\n", p.Name, p.Email, p.Age)
}
return nil
}
func main() {
// Insert some people
people := []*Person{
&Person{"[email protected]", "Joe", 30},
&Person{"[email protected]", "Lucy", 35},
&Person{"[email protected]", "Joey", 26},
&Person{"[email protected]", "Tariq", 21},
&Person{"[email protected]", "Dorothy", 53},
&Person{"[email protected]", "Joe", 35},
}
if err := InsertMultipleData(people); err != nil {
panic(err)
}
if err := ListAllValues(); err != nil {
panic(err)
}
}
When you executed the above code, you will see this result. you can see function was inserted the duplicated data and update that row with a new information, which you will see from last function. It isn’t applied idempotent concept.
Inserting ID: [email protected]
Inserting ID: [email protected]
Inserting ID: [email protected]
Inserting ID: [email protected]
Inserting ID: [email protected]
Inserting ID: [email protected]
All the people:
[+] Dorothy | [email protected] | 53
[+] Joey | [email protected] | 26
[+] Joe | [email protected] | 35
[+] Lucy | [email protected] | 35
[+] Tariq | [email protected] | 21
For implement the idempotence, I should add code for query as verify existing data before insert or update data.
Firstly, I just create new function to retrieving existing data.
func GetDataByID(key string) *Person {
// Create a read-only transaction
txn := db.Txn(false)
raw, err := txn.First("person", "id", key)
if err != nil || raw == nil {
fmt.Printf("[-] Not found %s", key)
return &Person{}
}
return raw.(*Person)
}
Then, I will update the InsertMultipleData
function to get data that will skipped if it found.
func InsertIdempotentData(people []*Person) error {
for _, p := range people {
existed := GetDataByID(p.Email)
if existed.Email == p.Email {
fmt.Printf("Skipping insertion for ID: %s (already exists)\n", p.Email)
continue
}
// Insert the new person
if err := InsertDataWithID(p); err != nil {
return err
}
}
return nil
}
If you replace the previous function call from InsertMultipleData
to InsertIdempotentData
, you could see result below. You can found 1 logging message that skipped insert operation.
[-] Not found [email protected] ID: [email protected]
[-] Not found [email protected] ID: [email protected]
Skipping insertion for ID: [email protected] (already exists)
[-] Not found [email protected] ID: [email protected]
[-] Not found [email protected] ID: [email protected]
[-] Not found [email protected] ID: [email protected]
All the people:
[+] Dorothy | [email protected] | 53
[+] Joe | [email protected] | 30
[+] Joe | [email protected] | 35
[+] Lucy | [email protected] | 35
[+] Tariq | [email protected] | 21
🐯 Idempotence is an important concept in programming, particularly when dealing with operations that modify state or data. An operation is considered idempotent if it can be applied multiple times without changing the result beyond the initial application. Idempotent operations are desirable because they enhance reliability, consistency, and simplicity in software designs. They ensure that the final state remains predictable and free from unintended side effects or data corruption, even if the operation is retried or repeated due to failures or other issues. 👻
Источник: dev.to
Наш сайт является информационным посредником. Сообщить о нарушении авторских прав.
go devops idempotent coding