Rivets alone are not designed to secure materials like cloth that can stretch and or tear out easily from the edges. One method to secure materials that stretch and or tear out easily from the edges to a surface using rivets or screws is to use a fender washer (oversized washer) under the head of a rivet or screw or to use a strip of sheet metal along the edge being secured with holes drilled through it for fasteners. This is to create a sandwich between material that stretches and or tears out easily from the edges to minimize the material tearing out.
The attached image is an example of a grommet kit that is used for repairing tarps and that people have also used to repair flags and maybe what you are looking for instead of rivets to help secure a flag. Grommets need to be installed a minimum distance from the edge of material to minimize tear out.