The A-polynomial is unlike other more famous knot polynomials in that it doesn't have a skein-theoretic definition. It is defined from a 3-manifold bounded by a torus (like a knot complement). One considers the $SL_2(\mathbb{C})$ characters of the fundamental group of the boundary which arise from representations of the fundamental group of the manifold, and finds that these define an algebraic curve in complex 2-space. As such, this curve is described by a polynomial in two variables -- the A-polynomial. Particularly attractive is the fact that there is no good algorithm for calculating the A-polynomial of all knots, and there are 'holes' in the knot table starting at eight crossings. To extend the definition of the A-polynomial to tangles, we want to identify what parts of the construction are functorial, and preserve pushouts or pullbacks. Then we can use the view of a tangle as a a cospan to take us most of the way to the A-polynomial. At this stage, we can define an algebraic structure which -- when we start with a knot -- can be used to extract the old A-polynomial. However, the calculations get very large very quickly. Without the use of a computer to manipulate the algebra for us, we cannot yet even verify that we do get back the right answer for the trefoil and figure-eight knot, let alone start filling the holes up at eight crossings. But the groundwork has been laid.