Computational Accelerator Physics Grand Challenge: Electromagnetics

We will address the electromagnetic calculations involved in modeling accelerator-driven applications and next-generation accelerators by developing a comprehensive set of codes capable of evaluating large complex accelerator structures and components efficiently and with high accuracy. This set will consist of eigenmode solvers, static solvers, and time domain solvers. In this approach the solvers will be fully three-dimensional, and will employ irregular grids to model structures using higher-order finite elements. They will be highly optimized for a variety of HPC platforms, and will utilize object-oriented techniques for reusability.

Eigenmode Solvers

Eigenmode solvers are essential for computing frequencies and fields of modes in cavities. Considerable groundwork in this area has been laid over the years by Kwok Ko and co-workers at SLAC, and by Eric Nelson of LANL. As an example, Eric Nelson has developed a 3D code called YAP which is able to compute cavity mode frequencies with relative error less than 10^(-5) and fields with relative error less than 0.3%. Presently this code runs on workstations so that high accuracy computations for multi-cell structures are not possible (for example, to model accurately a 200~cell DDS accelerator section will require about 50 GB memory).

The existing large sparse generalized algebraic eigensolver in YAP can be parallelized and improved in many ways. One promising route is to replace the standard conjugate gradient solver with a modified conjugate gradient solver that takes advantage of the definiteness of two sparse matrices A and B instead of simply combining them into an indefinite matrix A-lambda*B. Another promising route is to employ a Lanczos algorithm as a replacement for the subspace iteration method. Last but not least, preconditioning of the sparse matrices will be studied and employed.

Eigenmode solvers compute the harmonic source-free fields and the corresponding frequencies of a closed structure. Frequency domain solvers are similar, but they compute the response of a system driven at a specified frequency. Frequency domain solvers typically model systems that are not closed but instead have waveguide ports. Our approach here parallels the approach for the eigenmode solver. The only addition is the inclusion of waveguide port source terms and the solution of waveguide port eigenmode problems.

Besides the above-mentioned effort there has been a separate, ongoing effort at SLAC to implement eigenmode solvers on massively parallel platforms. This has been done in collaboration with Stanford University's Scientific Computing & Computational Mathematics (SCCM) program. Currently the code GOOFEE (developed by Xiao-wei Zhan of SCCM) calculates axisymmetric structures (2D) for arbitrary azimuthal order on the Intel Paragon computers at Oak Ridge National Laboratory. Analysis of the dipole wakefields in a 206-cell Detuned Structure (DDS without the damping manifolds) with GOOFEE is in progress. The code has also been used as a vehicle to study parallelization issues such as domain decomposition, preconditioning of sparse matrices, and the effectiveness of direct versus iterative methods.

The Grand Challenge project provides an excellent opportunity for these two complementary efforts to merge. The two codes share commonalities in both being finite element-based and written in the C++ language. The combined experience of the two labs, together with the participation from SCCM, gives the collaboration the essential ingredients to realize a 3D parallelized eigenmode solver in a reasonable time frame. Following this, further extensions to include periodic boundary conditions and lossy materials will be considered.

Static Solvers

Static solvers are used to compute the DC fields due to applied potentials or beam space charge (electrostatics), or due to magnetic focusing systems comprised of permanent magnets and currents (magnetostatics). These fields are involved in beam propagation and very often beam specifications place stringent requirements on the field errors. A major source of error comes from the inexact modeling of geometric complexities, especially when they are embedded in large domains. This can be remedied by local grid refinement (the h method), by increasing the order of the elements (the p method), or a combination of both (the hybrid h-p method).

We will develop finite element statics solvers that adopt the hybrid h-p method. Fast algorithms will be studied and evaluated to enable the solvers to run efficiently on HPC platforms such as the T3E (Stanford's SCCM program has considerable expertise in this area). Initially only linear material properties (dielectrics, permeable materials) will be treated. We will work on an interface that allows field data from these solvers to be interpolated accurately onto the particle grid used in the beam dynamics simulations.

Time Domain Solver

Time domain solvers are useful for studying rf propagation in accelerator structures without the beam, and for computing wakefields due to the transit of the beam. In either situation the treatment of boundary conditions is crucial if one is to model traveling waves correctly. As accelerator structures are driven via waveguides and are terminated in beampipes (also waveguides), a broadband matched boundary condition at these ports is absolutely necessary for simulating pulse propagation and beam-cavity interactions. Broadband waveguide boundary conditions have been implemented in time domain solvers based on regular grids and enable codes like MAFIA to calculate scattering parameters as well as beam impedances quite effectively. Recently, absorbing boundary conditions are coming into use in finite element codes to model exterior problems.

Since 1995, SLAC has pursued the effort to develop a time domain finite element solver to improve on the inaccuracies that existing codes incur when modeling curved surfaces with regular grids. A time leapfrog scheme and a narrowband excitation/absorbing boundary condition have been added to GOOFEE, and progress is being made in dealing with discretization errors and stability issues. For the Grand Challenge project, this work is being extended to three dimensions, and we will implement a broadband waveguide boundary condition to compare results with finite difference time domain codes.

To simulate beam effects, it will further be necessary to develop means to calculate wakefields with high accuracy. It is the experience at SLAC from modeling large electromagnetic structures that the narrow band impedances or resonances can be much more easily obtained with wakefield data than with eigenmode solutions, especially if the beam is short. This is because not all the eigenmodes couple to the beam, and if the beam spectrum overlaps a significant portion of a very dense mode spectrum, one quickly runs out of memory to store all the modes. Therefore a direct calculation with the beam is not only more physical but computationally easier as well. Besides loss factor and beam impedance, one can also obtain wall heating in a single computation. We will develop the numerical techniques required to evaluate wakefields accurately and efficiently on the T3E. They include, for example, wake integration along deformed contours and high-resolution FFT of the wake potential.

Back to Technical Information.

Back to main page.