An abstraction is "conceptual process where general rules and concepts are derived from the usage and classification of specific examples" wikipedia. A principle is a "proposition or value that is a guide for behavior or evaluation" wikipedia. I use the term pattern here as "the re-usable form of a solution to a (design) problem" (see also wikipedia).
Using abstraction and principles and patterns can be necessary (for instance in object-oriented software development to deal with its underlying deficiencies), but is generally very helpful. Quite often, applying the right set of them will help minimizing discontinuities avoiding cumbersome solutions. They are based on science, provided by some sort of authority, developed by best common practice (of a community or in some context), or selected using common sense and experience. Except for the scientific origin, hardly every will we find an explanations why those and not others are important. We can also find the negative, for example anti-patterns, See for instance wikipedia or this discussion on stackoverflow.
Here are a few things that I consider fundamental, based on my experience in science, research, and development. A lot of references and further discussion can be found in my research notes on Introduction to Network Architecure, see site and skb-web for details.
When looking at patterns, the best starting point is the work of Christopher Alexander, A Pattern Language (1977) and The Timeless Way of Building (1979). For networks, look at John Day’s Patterns in Network Architecture (2007). More recently, there is an interesting book by Jason McColm Smith called Elemental Design Patterns (2012). See rn-ina for more details.
This is fundamental. Everywhere. Not just in networks or software development. Its origin is in operating systems, namely by Hansen (Nucleus), and Wulf and Levin (Hydra) - see rn-ina for more details.
This abstraction (or principle) addresses the often essential problem, which is variance, not complexity. Using the principle (or abstraction) of separating mechanism from policy can (usually does) help to minimize variance. It does not necessarily minimize complexity.
To avoid reudctio ad absurdum, we can use a simple guideline: a mechanism focuses on the purpose (of something) and policies focus on ways to realize it. The goal then is to maximize invariance (mechanisms) and to minimize variance (policies). We need to look at commonalities and maximize them, thus reducing OPEX and CAPEX while improving management. Again, minimizing discontinuities also helps to avoid cumbersome solutions.
Use it wherever you can!
For me, these principles are fundamental. They are invented most and described all by Robert Martin, better known as Uncle Bob (wikipedia). There are five of them, as the abbreviation already suggests (wikipedia):
Single responsibility - A class should have only a single responsibility, that is, only changes to one part of the software’s specification should be able to affect the specification of the class.
Open–closed principle - "Software entities … should be open for extension, but closed for modification." (conceived by Brtrand Meyer)
Liskov substitution principle - "Objects in a program should be replaceable with instances of their subtypes without altering the correctness of that program." (devised by Barbara Liskov)
Interface segregation - "Many client-specific interfaces are better than one general-purpose interface."
Dependency inversion principle - One should "depend upon abstractions, [not] concretions."
These principles can be applied to more than just software development!
Principles that help to properly organize classes in larger systems wikipedia. Or text fragments into larger document. Or … so they are actually more fundamental, imho. These principles come in twpo groups with three members each:
Principles of package cohesion
Reuse-release Equivalence Principle - create packages with re-usable classes (all or none!)
Common-Reuse Principle - what is to be re-used together belongs to the same package
Common-Closure Principle - each package should not have more than 1 reason to change
Principles of package coupling
Acyclic Dependencies Principle - there should (i.e. must not) be any cycles in the dependency structure
Stable-Dependencies Principle - what is volatile should not be the basis for what is difficult to change
Stable-Abstractions Principle - stable should mean abstract, unstable should mean concrete (a bit similar to mechanism and policy, no?)
Every time we solve a problem it is related to a particular domain. These domains tend to come with their own context and models and terminology, or even taxonomy. The concepts of Domain-Driven Development (DDD) and Domain-Specific Languages (DSL) try to provide abstractions and more importantly patterns.
Martin Fowler defines a large set of DSL patterns in hos book Domain Specific Languages (book, pattern list, more information). There are four rather important patterns in this book:
Expression Builder - "An object, or family of object, that provides a fluent interface over a normal command-query API", pp343
Function Sequence - "A combination of function calls as a sequence of statements", pp351
Method Chaining - "Make modifiers return the host object", pp373
Object Scoping - "Place the DSL script so that bare references will resolve into a single object", pp385
The second important work is the book Language Implementation Patterns by Terence Parr (pragprog). While naturally heavily focused in ANTLR (developed by Parr), the book is an excellent source on how to write languages, write software that can automatically parse them, and write compilers. Any time you need to parse something, look at this work first!