DevOps and the DBA
How Do DBAs Fit In?
These three facets of DevOps can be applied to databases and database administration, though it does require database professionals to think differently about their tasks and their roles.
Database object deployment and development is the easiest area to apply these principles. When talking about developing for operations and managing responsibilities between teams, database professionals should be concerned with how database code is deployed. Usually this will mean the administrators will be involved in selecting tools for source controlling databases and building processes to deploy this code to the production environment. They should also work with the development teams to make sure code is written well and along agreed upon standards. However, this should not be approached in the traditional code review process, as this is much like the developer kissing the DBA’s ring for permission to deploy. Instead, DevOps DBAs should act as mentors and guides to build stable and efficient database code during the development phase.
Another element that DevOps DBAs should begin to consider is how they construct their environments. More and more commonly, servers and their binaries are considered disposable and stateless, and so the only stateful element of a database system is the data itself. Since most relational database systems offer data replication for high availability (such as Availability Groups for SQL Server and DataGuard for Oracle), DevOps DBAs have options to distribute their data in such away that they can replace server assets without affecting the data. This offers several strong advantages for maintaining asset
consistency and resolving server issues without affecting the uptime or response of the data systems. Naturally, there is a lot of detail we could go into around this, but I’m afraid I’ll have to save that for a future article.
This brings us to the biggest challenge of databases in a DevOps world: the data itself. Most DevOps processes consider assets, such as code and configs, as stateless and easily replaceable. Deploying code rapidly, as well as doing rollbacks, can be easily performed by simply overwriting these stateless assets. However, by its very nature, data should be considered stateful, and must be maintained. This means data professionals must marry DevOps principals with data stewardship and act, not only as data guardians, but shepherds that guide the process. Continue reading this article at Simple Talk
Automating the Database: A Win-Win for DBAs and DevOps
DBAs expend tremendous effort on trying to align the database with agile development and production, but a new approach is needed. The key is greater systematization and automation, ensuring uninterrupted consistency throughout the DevOps workflow – from task definition through application deployment.
Automated and enforced version control for the database is necessary for a single source of truth throughout development, preventing the instability that can break a deployment.
For developers, preventing database-linked errors at the coding or integration phases, as well as raising flags of potential conflicts and non-policy changes, at early stages of SDLC, allows a faster and much more efficient response. The DBA becomes less of a bottleneck for development teams.
For the DBA, problematic coding practices affecting the database are blocked automatically or trigger a warning, alleviating the need to review every bit of code by developers. This includes identifying undocumented hotfixes – source of configuration drift – implemented in the target schema at any point during development. This allows the DBA to focus on maintenance, planning deployments and providing needed guidance.
For management, this end-to-end, evolutionary database development process, along with open communication, direct reporting and enhanced collaboration internally, optimizes the development to production cycle and reduces its cost. Continue reading this article at InfoQ
Five DevOps Essentials for DBAs and Developers
A fundamental tension exists between development and operations/production DBAs:
- Development wants continuous change and enhancement, is focused on meeting schedule targets
- Production operations wants stability and controlled change, and is focused on meeting reliability targets
- In practice, the Production and Development groups work in their own silos, with separate tools, and are separately evaluated for successful completion of their mission.
Continue reading this article at LogicalRead
The role of the database within DevOps and what it means for the DBA
The evolution of the DBA
For years, IT departments did not have the reputation of being most cohesive unit – until DevOps came along. DevOps requires the sharing of information between the development and operations teams, which leads to a much safer and more efficient automated application delivery process. One of the consequences of this increase in volume of application releases and code updates was a greater workload placed on the database and its administrator, the DBA.
Responsible for manually implementing, recording, testing and managing all changes to the database, DBA’s experienced backlash from the development and operations teams, as they were blamed for delays in releases. This suddenly increased workload created a point of congestion, or bottleneck, as DevOps was automating processes and DBA’s were manually implementing changes.
The database subsequently gained the reputation of being the most vulnerable part of the application delivery process, often to the chagrin of the business executives, who were upset that glitch filled apps and delays in releases were causing them to lose money. Continue reading this article at ITProPortal
How the shift to DevOps affects application and system DBAs
Breaking down the silos that exist between development and operations and introducing a safety net that ensures the safe deployment of database changes are effective solutions for bringing database development into the modern, automated climate that is necessary for meeting consumer demands.
It is not necessary to interject automated processes with time-consuming manual analyses and builds. Under ordinary circumstances, it’s difficult to track who is making changes to the database. This makes it harder to create reliable releases, and can cause you to spend hours constructing release code.
Your process must support you in generating the correct upgrade SQL scripts for the structure changes and meta-data changes. This will save you hours of valuable time you’d otherwise spend manually generating, reviewing, testing, executing, or handling the rejects of the SQL scripts. A fail-proof safety net automatically detects configuration drift, determining which code is safe to deploy, and automatically protects the target environment when necessary. It will also identify when changes must be merged (and when human intervention may be needed) to ensure safe and speedy deployment.
Continue reading this article at DBMaestro
A DevOps DBA? Yep, they exist.
A database administrator, or DBA, is responsible for all activities related to the maintenance of a database environment. They design, implement and maintain every element of a database system. It is therefore surprising that often times – during DevOps implementation – database integration will be the last component to be implemented by a company. This can lead to problems during a turnover to DevOps, as developers lack the experience to perform the job of a DBA adequately.
On the other hand, the job of DBA is one that seemingly doesn’t fit on a DevOps team. It is a specialist function, after all, and DevOps only thrives when members of a team can act in many disciplines and perform many different duties. Continue reading this article at DBMaestro
It’s High Time DevOps Stopped Failing DBAs
The results of a recent survey called Assembling the DevOps Jigsaw (link is external) suggest that just 20 percent of businesses (and probably fewer) have fully implemented DevOps. Of course, there’s some contention over what constitutes “full implementation” and I’m here to tell you that even those advanced DevOps adopters are likely missing a key piece. Despite claims of “winning” at DevOps, a majority of businesses still haven’t married their database with other DevOps processes.
Somewhere along the path to continuous integration and so-called “full” DevOps deployment, the database got left behind in favor of speed through the build, test and deployment phases of application release. It’s great that dev teams have been able to automate development processes as much as they have. And the speed gains have been a boon for organizations making an effort to continuously release updates to meet customer needs. Continue reading this article at DevOps Digest
NO SUCH THING AS A DEVOPS DBA
Doing IT work is hard. Being a very good C# developer who has a full grasp of appropriate patterns, service methods, proper use of tools & tooling, and all the rest, that takes time. Let’s toss in learning T-SQL and database fundamentals on top of that. More time. Let’s also throw in server hardware and OS configurations and PowerShell to manage it all. While we’re at it, virtual machines and virtual machine management. Oh, and Azure. So, there are probably, a few people, who are legitimately good at all this stuff. But, it’s been my experience that most of humanity (myself included) are adequate at a few things, less than adequate but functional on a whole lot more, and, frankly, suck at everything else. And that adequacy assumes you work at it. So, all developers, by virtue of the magic of DevOps, are not only going to learn all of the above and more, but they’re going to be so capable that they’re able to appropriately respond in emergency situations with all of the above and more. No. Continue reading this
article at Grant Fritchey
Is DevOps crushing the DBA?
Traditionally, DBAs had all the power at least when it came to DBMS decisions. I’ve read many blogs recounting the nostalgic days when DBAs laid down the law to prevent rogue implementations. This was not always good for innovation and helping businesses respond to market opportunities. The arrival of Software-as-a-Service and the rise of mobile applications have completely changed the dynamic and companies holding on to such rigid IT models are being left behind. Agile is the approach of choice in Shoreditch and beyond among the trend-etting next-gen tech start-ups. Even the UK Government Digital Service has talked consistently about the need for becoming more agile.
We all accept that command control approaches to management are limiting. DevOps is giving companies the opportunity to introduce new applications, and adapt them dynamically in response to customer need. This is an immensely positive development for CIOs. Now they can be seen making a valuable contribution to business strategy, rather than simply sitting with arms folded and shaking their heads when the CEO asks for change. Continue reading this article at JaxEnter
Reinventing Yourself as a Next Generation DBA
“Developer Friendly” Technologies
In addition to changing IT management philosophies, developer-friendly technologies like ORMs (frameworks for working with relational databases) and NoSQL (database engines often considered to be “schema-less”) are leading some organizations to consider adopting the concept of NoDBA. This is quite literally the elimination of the DBA role as a function, but we have to ask ourselves why this is.
For some organizations, this may be a deliberate decision. For others, this is an artifact of the organization constraints placed on the application teams. They are simply acting in a way that aligns with their goal of “delivering the application to the business faster”. Developers (and development management) see these new technologies as a way to skip past their organization’s DBA team either entirely or significantly.
ORMs, as an example, shift the data modeling, schema management and query writing into a framework of objects and methods within the application itself. This leads developers to believe that they no longer require a DBA to be part of the application development process, because hey, it’s application code now, right? DBAs can just give the developer an empty database server and get out of the way, right? Continue reading this article at CumuLogic
It’s the data, stupid: Why database admins are more important than ever
“I think the roles of the developer, the DevOps guys, and data guy—maybe it’s a data engineer, maybe a DBA, maybe a data scientist—those roles have to cope with a myriad of new technologies,” said Gorman. “Each of those technologies has their own spectrum of maturity, features, and capabilities.” And that means that each of those roles now requires at least some of the skills of a DBA.
Whoever ends up in the DBA role for these systems doesn’t just need to have a general understanding of them—they need a much more nuanced understanding of what’s going on inside their systems than they may have ever needed with relational databases. Just as the behavior of SQL queries can be tuned to some degree for each relational database, getting the best performance out of newer non- relational systems requires DBAs to have a deep understanding of their inner workings.
“How a database ‘behaves’ is highly dependent on a few choices that the developer of the database makes at the lowest levels,” Lalonde said. “If you know what those choices are and how this particular database has made those choices, then you can get a good idea for how it will behave, generally speaking.” Continue reading this article at Ars Technica
SQL SERVER – DevOps for the DBA
DevOps is a collaborative approach to software development and delivery that is influenced by Agile development methods and Lean IT principles; the idea is toSQL SERVER – DevOps for the DBA – Notes from the Field #091 devops decrease time-to-market by looking at (primarily enterprise) software development as a life cycle, and finding ways to automate and increase efficiency. The general framework
is often described as consisting of the Three Ways (a phrase fully explored in The Phoenix Project) Continue reading this article at SQLAuthority
What Software Developers Need to Know About DBAs and DevOps
Not every development team is going to have a DBA.
In fact, many organizations are going to have developers—that’s you—doing many of the DBA duties, so it is useful to know your way around a database and some of the basics of installing and maintaining one.
For many software applications, the database is an important piece of the business, so whether the role of maintenance and administration falls directly into the hands of a DBA or is distributed amongst developers, it’s quite important that someone does the work.
Databases grow over time and can be pretty big resource hogs, so it’s critical to pick the right hardware to run a database on and to determine when it’s time to upgrade.
Databases also contain some pretty important data, so care has to be taken to back them up regularly, and someone needs to put together a disaster plan to restore a database or keep things running if one happens to fail.
And let’s not forget performance. Over time, databases which are not well designed or tuned can get slow and inefficient, so special care has to be taken to profile the database and determine how the data should be arranged or indexed to make things faster. Continue reading this article at Simple Programmer
Cassandra Tutorial: Setting up Ansible for our Cassandra Database Cluster for DevOps/DBA tasks
Ansible is an essential DevOps/DBA tool for managing backups and rolling upgrades to the Cassandra cluster in AWS/EC2. An excellent aspect of Ansible is that it uses ssh, so you do not have to install an agent to use Ansible.
This article series centers on how DevOps/DBA tasks with the Cassandra Database. However the use of Ansible for DevOps/DBA transcends its use with the Cassandra Database so this article is good information for any DevOps/DBA or Developer that needs to manage groups of instances, boxes, hosts whether they be on-prem bare-metal, dev boxes, or in the Cloud. You don’t need to be setting up Cassandra to get use of this article.
This was later split into two parts. * Part 1 * Part 2
The most up to date versions will be in the above two links.
Cassandra Tutorials: Tutorial Series on DevOps/DBA Cassandra Database
The first article in this series was about setting up a Cassandra cluster with Vagrant (also appeared on DZone with some additional content DZone Setting up a Cassandra Cluster with Vagrant. The second article in this series was about setting up SSL for a Cassandra cluster using Vagrant (which also appeared with more content as DZone Setting up a Cassandra Cluster with SSL). You don’t need those articles to follow along, but they might provide a lot of contexts. You can find the source for the first and second article at our Cloudurable Cassandra Image for Docker, AWS, and Vagrant. In later articles, we will use
Ansible to create more complicated playbooks like doing a rolling Cassandra upgrade, and we will cover using Ansible/ssh with AWS EC2. Continue reading this tutorial at Cloudurable
Data Scientist, DevOps Engineer and DBA Among Top IT Jobs for 2017
Also with a starting salary of $110,000 but with far fewer openings (2,725 as of this writing), is the title of DevOps engineer. With a job score of 4.7 and a job satisfaction rating of 4.2, DevOps engineers help organizations keep their application development and deployment effort on track.
In seventh place is the role of database administrator (DBA). Tasked with configuring, managing and maintaining the overall health of their organization’s critical business databases, DBAs command a starting salary of $93,000. Continue reading this article at Datamation
An Open Letter to Database Administrators: Be the DBA of the Future!
DBAs Must Partner with Development to Align Interests
Development thrives on change — it’s their job. Of course, change can be the enemy of stability and we all know stability is a friend of the DBAs. So how can development and DBAs achieve alignment in what seems to be an environment of competing interests?
Development requires near-instant feedback on changes as early as possible to deliver the RIGHT kind of change. This feedback increases stability. To provide this feedback, developers use unit tests to know at build time if their new code has failed. Thus, waiting until a database change reaches the production tier is unacceptable. DBAs need to provide feedback on proposed database changes earlier in the systems development lifecycle (SDLC). Waiting until a support ticket hits your team to review a change is not the solution; think proactive, be responsive. Continue reading this article at The New Stack
Say hello to Database DevOps Engineer
What is a Database DevOps Engineer?
Database DevOps Engineer enters and brings with him a plethora of skills ranging from low-level database maintenance, writing and optimising queries, knowing the planner inside-out; but also low-level operating system configuration, picking the right storage and optimising its parameters, taking care of the basic hardware monitoring, creating Munin/Grafana dashboards, writing Nagios/Icinga/Sensu plugins. And of course automating all that stuff using Puppet/Chef/Ansible/Salt, simplifying common tasks with scripts, handling partitioning, LVMs, virtualisation and so on.
A Database DevOps Engineer is, in many aspects, just like any other DevOps guy in the company, just with a “hint” of specialised knowledge on the database side of things. He not only manages databases and protects the data, but also facilitates the evolution of the data layer – by advising the best ways to share the data amongst applications (be it by replication, ETL processes, foreign data wrappers and so on) and by implementing the solutions chosen by other teams in timely and repeatable manner so that when a need arises for another setup like the one chosen, there’s already an module/role/playbook/recipe that other, non-DBA team members can use. Continue reading this article at Travelling SysOp
DevOps – Start of Collaboration
The manual approval step is the biggest hurdle to overcome with going to pre-production. Going to the leaders and telling them we want to remove it and not have anything to replace it will fail.
- Involve the DBAs earlier in the process, pre-production is just too late, we need to start the collaboration a lot earlier
- Check to see if there is a way to check the delta scripts for certain breaking changes, like drop tables, drop columns, etc. That is basically what we are doing now, there is no reason we couldn’t automate that
- Start making use of some sort of tool to verify database guidelines are being followed much earlier in the process, ideally during the build steps
Continue reading this article at Code Aperture
DevOps and Source Control
The topic of DevOps and and Agile are everywhere, but how often do you hear Source Control thrown into the mix? Not so much in public, but behind the closed doors of technical and development meetings when Agile is in place, it’s a common theme. When source control isn’t part of the combination, havoc ensues and a lot of DBAs working nights on production with broken hearts. Continue reading this article at DBA Kevlar
Is Your Organization Correctly Implementing DevOps for the Database?
The Implementation of DevOps for the Database
As the database was incorporated into DevOps with automation, DBAs could deliver applications faster, reduce downtime, and enforce compliance. Using specialized database tools such as enforced database source control, database build automation tools, and database verification processes, the database became a stable resource in the DevOps tool chain. DBAs are in the unique position of understanding the anatomy of the database, while acting as middleman between the developers and operations, making them critical to the implementation of an efficient and high performing DevOps
strategy. Optimizing the database for DevOps is a must for organizations. The reason is simple; a slow database produces slow results—which is bad for business. Continue reading this article at Developer.com
The 2020 DBA: A Look Into the Future
The next generation of DBAs will be less involved in the hardware and software stack. They will still need to be database intensive, but not exclusively. Instead, the DBA will focus on tasks like capacity planning, and they will need to know when to provision more servers and when to retire them.
Just as the definition of what a database is has become less clear, so too has the role of DBA. With the spread of DevOps, more people within an organization have greater knowledge of the database and what makes it tick. Given how much time people are required to spend with their own stacks of data, isn’t everyone becoming something of a DBA at heart? Continue reading this article at DZone
THE MYTHICAL MYSQL DBA
For one, the handoff from Amazon may have been less than smooth, lacking proper documentation and so forth. It could also be that the handoff went to less experienced DBAs or perhaps, those more versed in the legacy technologies of Oracle and much less in the free-wheeling open-source ones like MySQL. Other reasons could be failures in capacity planning, incomplete or incorrect systems integration, or simply misconfigurations in the load balancer, replication of database and memory settings. Continue reading this article at Scalable Startups
DBA, GROW THYSELF – MOVING AND SHAKING IN THE ERA OF DATA DOMINANCE
To a company, being able to reliably push out software updates, new products, roll out new features or content, fix bugs, and provide solid user acceptance testing is paramount. It increases both user and executive confidence in the business and IT as a whole, and it creates fresh sources of revenue and opportunity. The last thing any DBA wants is to be the bottleneck in that process. In all areas of development from a database perspective – creation of development/QA databases, refreshing from production, providing viable statistics and metrics, monitoring, and backup and recovery – the DBA must be as active and safe as they would be in production. At some companies, even development environments have change request procedures that must be followed by the DBA or developers. Continue reading this article at The Oracle Alchemist
DBAs: Don’t Be a Bottleneck in Agile Application Development
Most famously, this has led to a divide between developers and IT operations teams that is known as the “DevOps” crisis. But according to Mike Jones, vice president of OutSystems, a provider of an application development platform for building enterprise applications, that same issue now also applies to database administrators (DBAs). Continue reading this article at IT Business Edge
The real answer, of course, is engagement with data professionals along the lines of the devops movement. It’s the barriers between groups that cause much of the ceremony, few data groups are populated by devils. We’ve seen success from inviting DBAs to stakeholder meetings where they get to engage with the broad aims of an application. We’ve also had NoSQL projects where we’ve reached out to the DBAs, helped them learn about this new technology, and got valuable help in supporting data needs. So although the NoDBA strategy is sometimes appropriate, engagement is often better. Continue reading this story at Martin Fowler
The changing role of the DBA – a move towards DataOps
I’m laughing writing this as I see it happening time and time again. The DBA team have a gun to their head and they are being hit from all angles, poor SQL, new database technologies, new platforms, changes every 2 weeks etc… Those pesky devs are taking over, the penny has dropped with the infrastructure team who are changing their titles to DevOps and learning EC2 and Chef. Continue reading this article at Onomi
Storage and Database Collaboration
IT applications, architecture, and operations are in the midst of a fundamental shift. With the adoption of virtualization, service oriented architectures, and cloud computing environments, collaboration is a requirement for solving problems in a fast and effective manner. Organizations that do not collaborate well will spend much more time troubleshooting issues instead of improving the quality of their service. Ultimately, the organizations that achieve high levels of collaboration will innovate faster while providing higher levels of service. These companies will take customer share and revenue from their less nimble and more problematic competition. Start your collaborative journey now and don’t fall behind your competition. Don’t be left without your seat at the table. Continue reading this article at AppDynamics
How the DBA job is evolving for future?
Being a Database Technology Specialist for almost 16 years I get to talk in several community conferences, colleges and startups, Recently I was talking about the career prospects of being a DBA in a college in Bangalore, India, I was talking about what is it like being a DBA? As usual by the end of the talk I encourage audience to ask questions and one of the very anxious attendees asked me a very interesting question, “What is it like being a DBA in future with rapid advancement in Automation and DevOps ?”, The guy who asked me this question is a aspiring DBA and he was so worried about future
… so I thought let me write about this talk in my personal blog. Continue reading this article at MinervaDB
The Changing Role of the Modern Database Administrator
Greater involvement with architecture & planning. Thinking of “old school” DBAs often conjures up an image of someone hunched over a keyboard furiously trying to solve issues over backups, snapshots and slow performing queries. While these functions remain important, today’s DBAs find themselves as concerned with DevOps topics, ranging from capacity planning to application scalability, rather than being just involved with the lower layers of the software stack. Continue reading this article at Talena
Five DevOps Essentials for Better DBA and Developer Collaboration
At the core, a proactive database performance management approach requires communication, collaboration and integration across development and IT, with everyone focused on delivering exceptional end user experience. These are fundamentally the same principles that provide the foundation for many Agile, Lean and DevOps initiatives. While transforming an organization to embrace these principles is a journey that takes time, there are five key steps that can accelerate early successes:
- Provide developers direct monitoring visibility to test, staging and production servers
- Enhance collaboration by making developers self-sufficient problem-solvers
- Begin building performance into the development process by making it a functional requirement
- Establish shared metrics and a basis for equal access to metric reports across all teams
- Ensure that everyone in IT aligns with end-user expectations
Continue reading this article at AdFontess