<?xml version='1.0' encoding='UTF-8'?>
<codeBook version="1.2.2" ID="DDI-MCC-ARM-ROAD-MPR-2014-v1" xml-lang="en" xmlns="http://www.icpsr.umich.edu/DDI" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.icpsr.umich.edu/DDI http://www.icpsr.umich.edu/DDI/Version1-2-2.xsd">
  <docDscr>
    <citation>
      <titlStmt>
        <titl>
          metadata-arm-roads-v1
        </titl>
        <IDNo>
          DDI-MCC-ARM-ROAD-MPR-2014-v1
        </IDNo>
      </titlStmt>
      <rspStmt>
        <AuthEnty affiliation="Mathematica Policy Research">
          Kenneth Fortson
        </AuthEnty>
        <AuthEnty affiliation="Mathematica Policy Research">
          Randall Blair
        </AuthEnty>
        <AuthEnty affiliation="Mathematica Policy Research">
          Seth Morgan
        </AuthEnty>
        <othId role="Data collection and initial cleaning" affiliation="Government of the Republic of Armenia">
          <p>
            National Statistical Service
          </p>
        </othId>
      </rspStmt>
      <prodStmt>
        <producer abbr="MCC" role="Review of Metadata">
          Millennium Challenge Corporation
        </producer>
        <prodDate date="2014-08-14">
          2014-08-14
        </prodDate>
        <software version="4.0.9" date="2013-04-23">
          Nesstar Publisher
        </software>
      </prodStmt>
      <verStmt>
        <version>
          Version 1 (August 2014)
        </version>
        <notes>
          Approximately 360,000 rural inhabitants in 265 rural communities
        </notes>
      </verStmt>
    </citation>
    <docSrc>
      <titlStmt>
        <titl>
          Armenia
        </titl>
      </titlStmt>
      <prodStmt>
        <prodPlac>
          Transport (Trans)
        </prodPlac>
      </prodStmt>
      <notes>
        <![CDATA[The Millennium Challenge Account with Armenia (MCA) aims to increase household income and reduce poverty in rural Armenia through improved performance of the country’s agricultural sector. MCA planned to achieve this goal through an integrated, nationwide initiative to improve major components of rural infrastructure, focusing on rural roads and the irrigation system, supplementing these infrastructure improvements with training in agricultural practices. By improving living standards among rural residents, these investments could in turn lead to economic growth in rural areas and throughout the country as a whole.

However, MCC funding for the RRRP was put on hold indefinitely due to concerns about the Armenian government’s commitment to democratic governance, stemming from the government’s crackdown on protests following the last presidential election. As part of its own rehabilitation effort, the World Bank decided to fund the rehabilitation of some but not all of the road links included in MCA’s plans. For the roads it funded, the World Bank followed the road project designs developed by MCA, with some of the designs updated due to changes in conditions since designs were developed (about two years ago). The revised impact evaluation will estimate impacts by focusing primarily on road projects originally designed by MCA but now funded by the World Bank.]]>
      </notes>
    </docSrc>
    <notes>
      <![CDATA[Objective: 
     o Better access to economic and social infrastructure 
     
Estimated ERR: 26 percent
 
Activities: Rehabilitation of up to 943 kilometers of high priory rural roads in road lifeline network 

Beneficiaries: 
     o Approximately 360,000 rural inhabitants in 265 rural communities 
     o Rural transportation sector]]>
    </notes>
  </docDscr>
  <stdyDscr>
    <citation>
      <titlStmt>
        <titl>
          Rural Road Rehabilitation
        </titl>
        <IDNo>
          DDI-MCC-ARM-ROAD-MPR-2014-v1
        </IDNo>
      </titlStmt>
      <rspStmt>
        <AuthEnty affiliation="Independent evaluator">
          Mathematica Policy Research
        </AuthEnty>
        <AuthEnty/>
      </rspStmt>
      <prodStmt>
        <producer abbr="NSS" affiliation="Goverment of the Republic of Armenia" role="Fielded and cleaned the Integrated Living Conditions Survey">
          National Statistical Service of Armenia
        </producer>
        <software version="4.0.9" date="2013-04-23">
          Nesstar Publisher
        </software>
        <fundAg abbr="MCC">
          Millennium Challenge Corporation
        </fundAg>
      </prodStmt>
      <distStmt>
        <contact email="impact-eval@mcc.gov">
          Monitoring &amp; Evaluation Division of the Millennium Challenge Corporation
        </contact>
      </distStmt>
      <serStmt>
        <serName>
          Independent Impact Evaluation
        </serName>
      </serStmt>
      <verStmt>
        <verResp affiliation="Propensity Score Matching"/>
      </verStmt>
    </citation>
    <stdyInfo>
      <subject>
        <keyword>
          Armenia
        </keyword>
        <keyword>
          Roads
        </keyword>
        <keyword>
          transport
        </keyword>
        <keyword>
          impact evaluation
        </keyword>
        <keyword>
          matching
        </keyword>
        <topcClas>
          Armenia
        </topcClas>
        <topcClas>
          Roads
        </topcClas>
        <topcClas>
          impact evaluation
        </topcClas>
        <topcClas>
          transport
        </topcClas>
        <topcClas>
          matching
        </topcClas>
      </subject>
      <abstract>
        <![CDATA[The key research questions guiding our design of the RRRP evaluation are:
• Did rehabilitating roads affect the quality of roads?
• Did rehabilitating roads improve access to markets and social services?
• Did rehabilitating roads affect agricultural productivity and profits, and if so, by how much?
• Did rehabilitating roads improve household well-being for communities served by these roads, especially income and poverty?

As is frequently the case for large scale infrastructure projects, it was not feasible to randomly select the roads that would be rehabilitated in the MCA RRRP from amongst the eligible. Instead, MCA planned to fund projects based on estimated economic rates of return (ERRs). The ERR is calculated from several inputs, including the vehicular traffic, vehicle operating costs, and the cost of the project, among others. An ERR was estimated for each of the road links under consideration for rehabilitation, and in order to be funded, a road’s ERR had to meet or exceed 12.5 percent. However, the MCC Board subsequently made the decision to discontinue funding any further road construction and rehabilitation under the Compact due to concerns about Armenia's democratic governance. Starting in 2009, the Armenian government instead accessed loans from the World Bank to rehabilitate many road sections that were included in the RRRP plans before MCC discontinued funding, using and updating the road project designs developed by MCA-Armenia. These roads were selected based on estimated ERRs as well as other factors such as the size of the population who would benefit.

The evaluation of rural road rehabilitation uses a comparison group design. The evaluation design identifies the counterfactual by defining a comparison group of roads selected from those initially proposed by MCA that share similar characteristics as those selected by the World Bank, but that were not rehabilitated. The evaluators adopted a hybrid approach that matches road links based on a limited number of key observable characteristics and then uses regression modeling to control for other characteristics that may impact outcomes. The most fundamental criterion used in identifying potential comparison roads is that it is among the roads that were originally considered for rehabilitation by MCA, indicating that the road conditions are poor enough to consider rehabilitation. 

The data for this evaluation  come from Armenia’s Integrated Living Conditions Survey (ILCS), an annual household survey fielded by Armenia’s National Statistical Service (NSS). MCA funded a considerable increase in the rural sample, which was in part aimed at facilitating the RRRP evaluation. The sampling plans were developed and implemented based on the evaluation design described above. The evaluation includes households served by 27 roads that were rehabilitated with World Bank financing based on designs funded by MCA and 28 comparison roads.]]>
      </abstract>
      <sumDscr>
        <collDate date="2008" event="single"/>
        <collDate date="2011" event="single"/>
        <nation abbr="ARM">
          Armenia
        </nation>
        <geogCover>
          The analysis covers households in 107 communities served by 55 road links spread throughout the 10 regions of marzes (excluding the capital city Yerevan).
        </geogCover>
        <anlyUnit>
          Outcomes are measured at the household level.
        </anlyUnit>
        <universe>
          <![CDATA[The analysis covers households in 107 communities served by 55 road links spread throughout the 10 regions of marzes (excluding the capital city Yerevan), and the sample included in the analysis does not, in a statistical sense, generalize to a larger set of a communities. 

The ILCS is a nationally-representative survey of Armenian households. Additionally, to accommodate the road rehabilitation and associated evaluation plans, a special sample covering 82 planned road links was drawn. However, these were ultimately not able to be used in the evaluation and will not be covered in the public use file.]]>
        </universe>
        <dataKind>
          Sample survey data [ssd]
        </dataKind>
      </sumDscr>
    </stdyInfo>
    <method>
      <dataColl>
        <timeMeth>
          Based on annual, repeated cross-sectional surveys of households in 2007, 2008, and 2011.
        </timeMeth>
        <dataCollector>
          National Statistical Service of Armenia
        </dataCollector>
        <sampProc>
          <![CDATA[The core sample of the ILCS includes 768 enumeration areas, each containing 8 households for a total sample of approximately 6,100 households. Additionally, MCA funded an oversample of 216 enumeration areas in rural communities, an increase of approximately 1,700 households. The oversample is dedicated exclusively to communities served by roads in MCA's initial set of rehabilitation-eligible roads. Additional communities served by rehabilitation-eligible roads were selected into the core sample by chance, and a total of approximately 2,200 households served by the original, eligible project roads are in the sample each year. Because the ILCS does not use the same sample of enumeration areas in each year of the survey, different rounds of the survey may include different communities served by these roads.

The treatment group in our analysis comprises communities served by the 27 road links that were in MCA-Armenia's original RRRP plans, were ultimately rehabilitated in 2009 and 2010, and were covered by the ILCS from 2007/2008 to 2011. Unfortunately, ILCS data were unavailable for the relatively small number of communities affected by MCA-funded rehabilitation of the pilot roads. As a result, all 27 treatment links in the evaluation sample were funded with World Bank loans to Armenia. The comparison group in our analysis comprises communities served by the 28 road links that were in MCA-Armenia's original RRRP plans, were not ultimately rehabilitated with financing from the World Bank as of 2011, and were covered by the ILCS from 2007/2008 to 2011. Our data include outcomes measured before any roads were rehabilitated and after they were completed for both the treatment and the comparison group road links. All told, our analysis sample includes over 50 communities and 2,300 households in each of the treatment and comparison groups, for a grand total of 107 communities (55 in treatment and 52 in comparison) and 4,848 households (2,560 in treatment and 2,288 in comparison).]]>
        </sampProc>
        <collMode>
          The survey is administered in person, with enumerators asking respondents each question and noting the response on paper hard copy. Copies of the instruments used in each year included in the present study can be accessed here: http://www.armstat.am/en/?nid=378
        </collMode>
        <sources/>
        <weight>
          We use sampling weights constructed by the National Statistical Service that accounted for the number of households sampled and the population of communities surveyed in each road link. The total weight across households for each road link is the same for all links. Household weights were rescaled to give each road link equal weight. The exceptions are when two links are actually connected and serve the same set of communities, in which case those links are combined into road link pairs. In such cases the road link pair has double the weight of individual road links.
        </weight>
      </dataColl>
    </method>
  </stdyDscr>
  <dataDscr/>
</codeBook>
