Sticking to decision problems simplifies the discussion of hard problems.

All computational problems can be made into decision problems.

Polynomial time algorithm: For algorithm f(x) there is a constant k such that worst case runtime of f on input x of size n is O(n^k).

P: Set of problems which have a polynomial time algorithm f(x).

NP: Set of problems which have a polynomial time verifier g(x,y). Algorithm g(x,y) verifies that the answer is "true" on x provided an oracle gives it a correct hint y.

Many problems in NP are shown to be at least as hard as SAT by reduction of SAT to the problem. Examples: vertex cover, independent set, clique, TSP, HAM Cycle, Integer linear programming, knapsack, bin packing.

======================================================

Cook's theorem: Every problem in NP can be polynomial-time reduced to SAT.

Proof sketch: Suppose you have a polynomial time algorithm S(c) to solve Circuit-SAT: It takes a circuit c of size m and determines if there is a setting of the input bits to the circuit that turns the output bit on. It does this in O(m^j) time for some constant j.

Suppose problem A, being in NP, has verifier g_A(x,y) which runs in n^k steps.

Build a circuit C that consists of n^k instances of a CPU chained together. Let a be the ciruit size of the CPU, a constant. The variables of the circuit are the bits of the hint y. For a given input x, the bits of x are constants. The circuit is satisfied if the output is "true", i.e. if the memory bit storing the output value gets turned on.

Apply S to C. S(C) runs in polynomial time O((a*n^k)^j), because the input circuit size is m = a*n^k.

Now g, with help from S solves problem A in polynomial time:

Solve problem A in polynomial time with this function

f_A(x){

1. Build C from verifier g_A.
2. Oracle input y is determined by running S(C).
3. Return g_A(x,y);

Corollary: If any NP complete problem is in P then NP = P.

There are thousands of NP Complete problems. None have been shown to be in P nor proven to be not in P.

======================================================

The curious case of linear programming (LP)
Input: set of linear inequality constraints on variables x, objective function F(x), bound B.
Output: "true" if and only if for some x, f(x) >= B.

The simplex algorithm is effective in practice, but exponential in the worst case. Karmarkar's relatively recent algorithm shows LP to be in P.

Note: LP is not integer linear programming (ILP). We showed ILP to be NP-Complete.

======================================================

Some problems are strictly harder than any NP Complete problem.

Example, The proper polynomial ideal question:
Given n polynomials f₁, f₂, ..., f_n in n variables x = (x₁, x₂, ..., x_n),
do there exist polynomials g₁, g₂, ..., g_n such that
1 = g₁(x)f₁(x) + g₂(x)f₂(x) + ... + g_n(x)f_n(x)?

This problem is "exponential space complete".

======================================================

Some problems are impossible.

Example, The halting problem: Given the text of a C program p and an input x for p, determine if p halts when run on x.