The COVID-19 pandemic is not only a health crisis but severely affects the mental, social and economic well-being of hundreds of millions of people. Poverty, unemployment, violence, isolation, depression etc are the effects of the pandemic and the counter measures to contain the outbreak. In this paper I want to discuss this social impact of the pandemic from the perspective of a capability based theory of social justice. In the first part of the paper I briefly present the social impact of the pandemic and how it disproportionally disadvantages certain social groups. In the second part I sketch a capability based theory of justice, which is primarily concerned with protecting vulnerable groups and guaranteeing them a sufficient minimum of basic capabilities. Then I will discuss ways to define and identify vulnerable groups and their needs. Final part of the paper I will discuss how such a theory of justice could inform policies during and after the pandemic.