An embodiment of the invention is a multipoint conferencing system having the ability to dynamically assign an endpoint to a "closest" one of multiple conference servers is disclosed. In one embodiment, when an endpoint first connects to the conferencing system, its Endpoint ID (e.g., IP address, E164 address) is determined. Based on the Endpoint ID, the conferencing system selects the most appropriate ones of the conference servers to act as the endpoint's "local" conference server. That is, the endpoint will transmit and receive conference data from other endpoints via the selected conference server. If no other endpoints across the network happen to be viewing a particular endpoint's video stream, the stream is not provided to the network, thus conserving network bandwidth. An embodiment further creates appropriate linking between conference servers automatically, manages conference server topology and determines link type (e.g., Unicast vs. Multicast linking) automatically.