Methodology

Stage 1: Preparing

3. The JA Team (JAT) assembled by the Governance and Justice Group (GJG) comprised: a judge, economist, police officer and four lawyers.¹ Bluhm Legal Clinic comprised its Director and four law students.² Justice Mapping Center comprised its Director and associate³ supported by a team of five software programmers and designers.

4. GIZ appointed a project team to co-ordinate the work on the ground and liaise with the JAT and Abu Ahmed Jamadar, National Programme Director (NPD) to secure meetings and national-level institutional data.⁴

5. The JAT collected and reviewed a library of documents concerning the criminal justice system in Bangladesh. A full set of the documents is in the Library and includes a link to the Laws of Bangladesh in the Ministry of Law, Justice and Parliamentary Affairs (MoLJPA).

6. In close consultation with the GIZ office, consent letters were obtained from the Ministers of MoLJPA and Home Affairs as well as the Inspectors General of Police and Prisons and Chief Justice of Bangladesh through the good offices of the NPD in the MoLJPA.

7. GIZ contracted five District Focal Points (DFP) to assist the JAT gather institutional data in each district.⁵ Each DFP was selected on the basis of his/her standing in his/her district and knowledge of the criminal justice system.

Stage 2: Framing

8. The Citizen Survey was designed and a firm of enumerators contracted.⁶ The questionnaire was reviewed by two focus groups, one male and one female, of experts on the Bangladesh justice system, field tested and refined. The results of the Citizen Survey (with the questions) have been disaggregated for each district and are located in the Baseline Data.

9. Five separate, identical surveys were conducted simultaneously in the five districts. The same methodology, outlined below, was used for all five surveys.

10. Each survey was conducted through in-persons interviews with a random, representative sample of 1,200 citizens (6,000 in total), age 18 and up. The fieldwork was conducted over twenty-one days (Nov 15 – Dec 6) by Evaluation & Consulting Services Ltd (ECONS).

Representativeness of the Samples:

11. The population randomly selected for each district was designed to be representative of each district as a whole. The gender breakdown of the respondents essentially reflects the gender breakdown of the 2011 census for the districts surveyed:

	Male		Female
	2011 Census	Survey	2011 Census	Survey
Comilla	47.80	47.92	52.20	52.08
Gopalganj	49.29	49.25	50.71	50.75
Madaripur	49.28	49.25	50.72	50.75
Mymensingh	49.69	49.92	50.31	50.08
Rangpur	50.11	49.50	49.89	50.50

12. The urban:rural breakdown was also representative of the population, as described in the 2011 census. However, due to time and travel constraints, it was necessary to classify the areas categorized as "peri-urban" in the 2011 census as 'rural' in the survey.

	Rural		Urban
	2011 Census	Survey	2011 Census	Survey
Comilla	86.87	86.83	13.13	13.17
Gopalganj	92.71	92.67	7.29	7.33
Madaripur	88.98	88.92	11.02	11.08
Mymensingh	89.14	89.08	10.86	10.92
Rangpur	86.55	86.58	13.45	13.42

Margins of Error:

13. The margin for error for each survey was calculated to be 2.83% at a 95% confidence interval. Some questions were asked only to individuals who had experience with crime or the justice system, which will have an impact on the margin of error. Questions with less than 1,200 respondents are indicated in the report.

Sampling Methodology:

14. The basic sampling methodology used was multistage cluster and interval method, as described in more detail below:

Sampling Points Selection

• Rural Selection:
  • Due to travel constraints caused by elections-related political unrest, the number of sampling points needed to be restricted.
  • For each district, 1/3 of the upazillas were selected by using a proportional to the population systematic random sampling method.
  • The same process was then used to select two unions in each upazilla and two Moza (cluster of villages) in each union.
  • Once the Mozas were selected, the number of interviews assigned was weighted according to the proportion of the population.
  • The selected areas and number of samples selected are set out in Annex 1.
• Urban Selection:
  • No detailed reliable data on the economic stratification of the urban areas could be found.
  • A person familiar with each urban area was requested to prepare a list of the names of all areas, and indicate whether they were predominately poor, middle income, or rich.
  • One area from each category was then selected using a random number generator.
  • The numbers of respondents per category were assigned based on what locals estimated to be the economic breakdown of district urban centers: 43% low income, 47% middle income, and 10% high income.
  • The selected areas and number of samples selected for each are set out in Annex 2.

Household Selection

• Households in each Moza and urban area were selected using an interval method.
• Due to an absence of household list, each morning the survey supervisor selected a) a random spot in the center of the sampling area and b) two numbers from a hat: one between 1 and 5 and one between 6 and 10.
• The male enumerators were assigned the number between 1 – 5, and were instructed to use that number as their household starting point (i.e. if they were assigned the number 3, that day they would begin on the third house). Female enumerators were assigned the number between 6 and 10.
• Teams of one male and one female enumerator were then assigned a direction in which to start by the supervisor, and were instructed to continue on that path. If the path ended, they were instructed to turn right.
• Following the first selected household, the survey teams selected every 10th household on their path until the target number of surveys was reached.

Respondent Selection

• Cultural norms heavily influenced the process of respondent selection.
• Male enumerators were instructed to interview the most senior male member of the household and female enumerators the most senior female member of the household.
• Only one interview was conducted in each household.

Respondent/Household Substitution

• If the respective head of household was not willing to participate, the enumerator was instructed to thank the person and continue with the intervals of 10 to select the next household.
• If the respective head of household was not home at the time of interview, a request was made to speak with the next most senior person of the same gender.
• If that person was unable or unwilling to speak, an appointment was made to speak with the head of the household.
• In the event that the head of the household was not available at the time of the appointment or was unwilling to speak at that time, the enumerator was instructed to repeat the same process in the house directly to the right.
• In the event an enumerator team exhausted the number of households to sample in a Moza, they were instructed to continue on their same path into the next Moza.

Quality Control:

Training

• All of the enumerators selected had a minimum of four years of experience and received five days of intensive training (Nov 9 – 13), including two days of field testing, and were provided a training manual to take to the field.

Collection

• Supervisors were assigned to a team of four to six enumerators, and were required to directly witness interviews and spot-check the responses
• Each district was supervised by a Monitoring Officer, who was required to directly witness interviews and back-check the responses
• The political unrest prevented the rest of the research team from going to the field

Data Entry

• All of the data was double entered by two different data entry operators (error .04%)
• Cleaning and spot checks of the data entry process was supervised by a GIZ consultant

15. The Practitioner Survey (PS) was designed as a set of structured interviews with key informants. The PS is aimed at criminal justice practitioners working in the five districts and includes: police, lawyers, NGOs, prosecutors, judges, magistrates, court clerks, nazir, members of the public outside courts, UP Chairmen, UP Secretaries, Village Court Assistants, members of the public outside the VCs, Shalishkars, and prison officers.

16. The political situation in the country was tense prior to the January elections. Throughout November and December the unrest grew: hartals were frequent and extended to country-wide blockades of roads and other transport routes. By way of response and in consultation with GIZ and the NPD, the plan to deploy the JAT in the five districts was abandoned.

17. The five District Focal Points (DFPs) contracted to support the JAT on the ground were brought to Dhaka for consultation. The Practitioner Survey questions were tested on the five DFPs, amended and translated. The DFPs were then briefed on conducting the Survey in line with international standards and the principle of informed consent and provided with a set of data collection forms for each institution. A manual was drafted to assist as a Field Guide.

Stage 3: Collecting

18. The DFPs deployed at the end of November and were given two weeks to collect the institutional data and carry out the Practitioner Survey. In addition, GIZ deployed two interns to support the DFPs in the two largest districts, namely Comilla and Mymensingh. The interns were recalled by GIZ to Dhaka after a short time in the field after one had a 'cocktail bomb' thrown at him. The JAT called in each morning to each DFP to provide support and monitor progress.

19. The Bluhm Legal Clinic Director and four students supplemented the JAT to input the institutional data collected from the five districts. GIZ interns translated the returns from the Practitioner Surveys as they came in. The JAT then uploaded the data and returns from the PS into excel sheets. The responses were then consolidated across the five districts and can be found in the Baseline Data.

Stage 4: Discovering

Data Management

20. Each institution was asked in what format data was collected: manually or digitally.

• Courts: presented a mixed picture. Most districts reported their data was collected manually.
  • Mymensingh/Rangpur: all courts collected manually (ie in registers);
  • Comilla: all courts collected manually save for the Chief Judicial Magistrate's courts (CJM) which are digital;
  • Gopalganj: all courts collected manually save for the District and Sessions Courts (DSC) which have both digital and manual recording.
• Police: all thanas record data digitally save for Fulbaria and Issorgonj (Mymensingh).
• Prosecution has no national system in place.
• Prisons: only Mymensingh responded to this question, they operate both manual and digital systems.

21. Whether collected manually or digitally courts, police and jails report to the capital in hard copy.

22. None of the data in any of the institutions is disaggregated in terms of:

• gender
• type of offences (courts)
• bailable/non-bailable offences (prisons)
• age of offender

23. There is also high reliance on 'Other' as concerns both types of offences and methods of disposal.

24. A comparison of national data supplied by the Supreme Court and Prisons HQ shows significant disparities in places (even allowing for different dates) and in many instances the data do not add up.

25. Institutions were asked how they tracked cases.

• Cases are filed in the main in the courts by year. They are not tracked according to gender or age.
• The reporting of data up the chain to the national capital is by hard copy (courts/police/Mymensingh jail) and the regularity varies between monthly, quarterly, bi-annually or annually in the courts and daily or monthly (police) and weekly/monthly (Mymensingh jail).
• The institutional section responsible for the collection and analysis of national data (ie in Dhaka) did not appear to have the data readily available. Data requested by the JAT had to be requested directly from the district concerned.

26. The institutional data was checked and cross-checked in Dhaka. Gaps and queries were followed up with the DFPs in the five districts. The institutional data was then sent back to the DFPs to return to the institutional sources of data to check the data sets collected for each.

27. GIZ organized the collection of data from the heads of each national institution (courts, police and prisons) and forwarded the responses to the JAT. The results of the Citizens Survey were then analysed.

28. A meeting was held in England in January with the whole team and representatives from GIZ to review the data for each district and identify the story lines that emerged from a triangulation of the institutional data with the Citizen and Practitioner Surveys.

Stage 5: Analysing

29. Following this meeting, the JAT worked from home bases to work the data into formatted excel worksheets. This formed the Baseline Data for each institution in each of the five districts – see Baseline Data.

Stage 6: Designing

30. Key 'stories' were then extracted from the baseline data for the Design team to visualize in the web application and software programme that allows the information authority in Bangladesh that will house the databank to update the JA at regular intervals going forward.

31. A commentary was drafted to supplement the visualisations in each screen. The purpose is to highlight key issues suggested by the data. At the same time, Data Notes flag up where the data appears inconsistent or confusing.

Stage 7: Validating

32. The Justice Audit of the Five Districts was presented to the Ministry of Law, Justice and Parliamentary Affairs on 23 September 2014.

33. The Minister was then invited to nominate an 'authority' to whom to hand over the data and software programme to facilitate the application of the JA methodology going forward.