Observations of circumbinary planets orbiting very close to the central stars have shown that planet formation may occur in a very hostile environment, where the gravitational pull from the binary should be very strong on the primordial protoplanetary disk. Elevated impact velocities and orbit crossings from eccentricity oscillations are the primary contributors to high energy, potentially destructive collisions that inhibit the growth of aspiring planets. In this work, we conduct high-resolution, inter-particle gravity enabled N-body simulations to investigate the feasibility of planetesimal growth in the Kepler-34 system. We improve upon previous work by including planetesimal disk self-gravity and an extensive collision model to accurately handle inter-planetesimal interactions. We find that super-catastrophic erosion events are the dominant mechanism up to and including the orbital radius of Kepler-34(AB)b, making in situ growth unlikely. It is more plausible that Kepler-34(AB)b migrated from a region beyond 1.5 AU. Based on the conclusions that we have made for Kepler-34, it seems likely that all of the currently known circumbinary planets have also migrated significantly from their formation location with the possible exception of Kepler-47(AB)c.