1 Introduction

This code shows the process of how we extract the sample paragraphs as presented in Table 3 of the main paper.

2 Preparation

Loading libraries.

library(dplyr)
library(knitr)
library(kableExtra)

3 Paragraph extraction

Extract paragraphs that are predicted using our 4 models.

3.1 Content models (Economist vs. Non-economist)

Extract sample paragraphs from the validation datasets that are predicted using 2 reasoning models.

eco_content <- read.csv("./data_input/eco_content_validataion_para_prediction_result.csv")
noneco_content <- read.csv("./data_input/noneco_content_validataion_para_prediction_result.csv")
eco_clarity <- read.csv("./data_input/eco_clarity_validataion_para_prediction_result.csv")
noneco_clarity <- read.csv("./data_input/noneco_clarity_validataion_para_prediction_result.csv")

eco_content_short <- eco_content %>% select(question_index, paragraph_clean, content.scale.avg, content_label, rf_predict)

noneco_content_short <- noneco_content %>% select(question_index, content_label, rf_predict)


content_common_para <-
  inner_join(eco_content_short, noneco_content_short, by = "question_index", 
             suffix = c(".eco", ".noneco")) %>% arrange(rf_predict.eco)

content_common_para %>% head() %>% kbl() %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
question_index paragraph_clean content.scale.avg content_label.eco rf_predict.eco content_label.noneco rf_predict.noneco
5_6_97 The Bank of Japan has continued its policy of balance sheet expansion. The money base in Japan has increased by ¥30 trillion since the start of the year, in line with its target of an ¥80 trillion expansion over the year. -1.7559148 low 0.14 low 0.2233333
5_2_77 The available data suggest that the domestic economy continued to grow at a below-trend pace in the March quarter. Dwelling investment and resource exports appear to have continued growing strongly and there is evidence that the growth of household consumption has been gaining some momentum over the past six months or so. However, investment in the mining sector is declining noticeably and non-mining business investment remains subdued. Moreover, indicators of non-mining business investment intentions suggest that a significant pick-up is not in prospect over the next year or so. -1.3552619 low 0.32 low 0.4133333
3_6_49 Overall, the ESEA economies are highly competitive as low-value manufacturing destinations because of their relatively low labour costs, particularly relative to China (Graph 14). This, combined with recent policies to attract foreign investment, improve infrastructure and expand access to export markets, has driven the shift of lower-value global manufacturing to these economies; this is likely to continue (The Economist 2018). For example, Vietnam has attracted some of the footwear, clothing and textiles production from China over the past two decades. This has been assisted by the Vietnamese authorities lowering corporate tax rates and establishing an extensive set of bilateral trade agreements, including with the United States in 2001, Japan in 2007 and the European Union in 2018. 0.6944551 high 0.32 high 0.3966667
3_5_38 Business loan and deposit exception fees increased during 2016, but this was driven by changes in reporting methodologies. Abstracting from this factor, exception fees were little changed. -0.4018826 low 0.36 low 0.3766667
1_3_154 And third, as the Governor highlighted yesterday, we are opening up direct RTGS access to a broader range of firms. Five non-bank payment service providers now hold accounts in RTGS, and have seen benefits including faster transaction times and lower reduced individual transaction costs, and around twenty further firms are exploring the possibility of joining. -0.7889572 low 0.39 low 0.4433333
5_9_22 But the changes also resulted in an effective tax rate of more than 100 per cent on savings, which is hard to justify. Middle-income earners should get at least some reward - in terms of additional income - from savings. 0.5195058 high 0.55 high 0.4200000

3.2 Clarity models (Economist vs. Non-economist)

Extract sample paragraphs from the validation datasets that are predicted using 2 readability models.

eco_clarity_short <- eco_clarity %>% select(question_index, clarity.scale.avg, clarity_label, rf_predict, paragraph_clean)

noneco_clarity_short <- noneco_clarity %>% select(question_index, clarity_label, rf_predict)


clarity_common_para <-
  inner_join(eco_clarity_short, noneco_clarity_short, by = "question_index", 
             suffix = c(".eco", ".noneco")) %>% arrange(rf_predict.eco)

clarity_common_para %>% head() %>% kbl() %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
question_index clarity.scale.avg clarity_label.eco rf_predict.eco paragraph_clean clarity_label.noneco rf_predict.noneco
5_2_37 -1.0065032 low 0.24 The mining sector’s share of the capital stock has doubled since the early 2000s, driven by large-scale investment geared towards expanding the sector’s productive capacity. Based on a set of simple assumptions, mining investment’s share of GDP is likely to converge to between 2½ and 4 per cent in the long run, which is above the share that prevailed prior to the boom. Mining investment is expected to be relatively subdued over the next few years, because firms have limited appetite for further expansion. Instead, sustaining capex is likely to take on more importance as firms look to maintain their newly expanded productive capacity. Analysis based on company reports, the Bank’s liaison program and information from other data providers suggests that sustaining capex for Australia’s three major resource commodities will make a modest contribution to nominal GDP growth over the next five years, contributing around 0.2 percentage points per annum on average. low 0.288
1_4_53 -0.8594000 low 0.30 One of the questions underpinning this conference asks what is special about large, long-term asset owners? As a conceptual starting point, one might consider them endowed with some competitive advantages, and that availing of these could also serve a dual purpose in supporting financial stability and economic growth. For instance, their relatively stable risk preferences might empower asset owners to lean against excessive swings in risk premia, thus conferring a stabilizing influence on market cycles. More tolerance for short-term volatility could translate into bearing the types of risks that other investors pay a premium to avoid. And asset owners with long-dated liabilities should also be natural suppliers rather than demanders of liquidity, an approach that can be both individually profitable and helpful in stabilizing markets during disorderly conditions. low 0.436
1_10_173 -0.4917594 low 0.35 RBS may now feel a tinge of regret. After a £9.3bn merger with Vantiv, an American peer, last year, Worldpay is the world’s largest acquirer. It says it processed over 40bn transactions in 2018. Other non-banks, such as Global Payments, have also become giants (see chart). This is partly due to organic growth. Acquiring is more profitable than other processing jobs, which have become commoditised. It is also cheaper and faster to scale on the web: installing card terminals in-store requires labour and local presence. It helps that the volume of transactions, on which acquirers levy a fee, is rocketing, propelled by voracious spending in emerging economies. low 0.352
4_9_74 -0.5193228 low 0.38 The outlook for the global economy is a little softer than at the time of the May Statement. World GDP growth is expected to remain close to average over the rest of this year, but with annual average growth around 3 per cent for 2013 reflecting weaker growth around the turn of the year. Growth is then expected to pick up, to be slightly above average in 2014. Economic growth is expected to be stronger for Australia’s major trading partners than for the world as a whole. The slightly weaker outlook than a few months ago reflects, among other things, the assessment that growth in China is now unlikely to pick up much, if at all, in coming quarters. Rather, it is expected to remain at a pace that is close to the official target. low 0.412
4_9_168 -0.7311226 low 0.51 Further doubts stem from the leverage that has been granted to Japanese negotiators. They were brought to the table after America walked away from the tpp by the threat of tariffs on cars and car parts. Now they have concessions they can roll back if the Trump administration enacts those. Threats have worked once. But they could be less use in securing the big concessions needed if this supposed staging post is not to become the final destination. low 0.544
4_3_14 0.4219421 high 0.58 In the last few years there’s been some progress. Sydney in particular has started to add materially more medium high density along its major transport corridors. It’s probably not enough to unwind the accumulated backlog of a decade of policy inaction, but at least it’s in the right direction. But today’s record level of housing construction is the bare minimum needed to meet record levels of population growth driven by rapid migration. And public resistance is growing, partly because many of the policy changes were made without an extensive public discussion of their rationale. high 0.532

4 Output

Extract original survey scores for those paragraphs, and generate the final output table to compare the original survey scores and model prediction results. The final output table is shown as below:

## Extract the survey prediction results
eco_content_score <-
  eco_content %>% arrange(rf_predict) %>% 
  select(question_index, eco_content_score = rf_predict)


eco_clarity_score <- 
  eco_clarity %>% arrange(rf_predict)  %>% 
  select(question_index, eco_clarity_score = rf_predict)


noneco_content_score <-
  noneco_content %>% arrange(rf_predict) %>%
  select(question_index, noneco_content_score = rf_predict)

noneco_clarity_score <-
  noneco_clarity %>% arrange(rf_predict)  %>%
  select(question_index, noneco_clarity_score = rf_predict)



#extract the original survey results
survey_response_score <- read.csv("./data_input/survey_table_final.csv")

survey_response_score$question_index <- 
  paste(survey_response_score$survey_group, survey_response_score$question_group, survey_response_score$index.x, sep = "_")


sample_para <-
  survey_response_score %>% filter(question_index %in% c("1_1_186","4_7_21","1_7_82","2_2_14",
                                                       "5_5_30","3_8_112","1_8_40","1_1_111"))%>%
  select(question_index, source, Content, Clarity, record_score_by, paragraph = para, content.scale, clarity.scale)


#join the two scores together for each paragraph
sample_paragraphs <-
  sample_para %>% left_join(., eco_content_score, by = "question_index") %>% 
  left_join(.,eco_clarity_score, by = "question_index") %>%
  left_join(.,noneco_content_score, by = "question_index") %>%
  left_join(.,noneco_clarity_score, by = "question_index")

#take a look of the sample data
sample_paragraphs%>% head() %>% kbl() %>%
  kable_paper() %>%
  scroll_box(width = "100%", height = "200px")
question_index source Content Clarity record_score_by paragraph content.scale clarity.scale eco_content_score eco_clarity_score noneco_content_score noneco_clarity_score
1_1_111 6_smp_boxes_06_19 1 1 Non-economist Looking at more detailed data on cross-border bank lending from the Bank for International Settlements, it is evident that cross-border lending by European banks both increased most rapidly going into the crisis and subsequently contracted most sharply. Given that financial stress was concentrated in industrialised economies it is also noteworthy that lending to other industrialised economies peaked earlier than lending to emerging markets, which was curtailed only much later into the financial turbulence (Graph C2). This pattern is also evident in the sharp reversal of (net) flows between the United States and the United Kingdom as a result of reduced cross-border lending by European banks headquartered in London as institutions sought to unwind their exposures. -1.5434873 -1.5434873 NA NA NA 0.284
1_1_111 6_smp_boxes_06_19 3 3 Non-economist Looking at more detailed data on cross-border bank lending from the Bank for International Settlements, it is evident that cross-border lending by European banks both increased most rapidly going into the crisis and subsequently contracted most sharply. Given that financial stress was concentrated in industrialised economies it is also noteworthy that lending to other industrialised economies peaked earlier than lending to emerging markets, which was curtailed only much later into the financial turbulence (Graph C2). This pattern is also evident in the sharp reversal of (net) flows between the United States and the United Kingdom as a result of reduced cross-border lending by European banks headquartered in London as institutions sought to unwind their exposures. -0.1142080 -0.3721042 NA NA NA 0.284
1_1_186 10_economist 4 1 Economist The big question is whether we should expect these quirks to endure. Once a way to make above-market returns is identified, it ought to be harder to exploit. “Large pools of opportunistic capital tend to move the market toward greater efficiency,” say Messrs White and Haghani. For all their flaws and behavioural quirks, people might be capable of learning from their costliest mistakes. The rapid growth of index funds, in which investors settle for an average return by holding all the market’s leading stocks, suggests as much. 0.5642168 -1.5811388 0.9 NA NA NA
1_1_186 10_economist 5 2 Non-economist The big question is whether we should expect these quirks to endure. Once a way to make above-market returns is identified, it ought to be harder to exploit. “Large pools of opportunistic capital tend to move the market toward greater efficiency,” say Messrs White and Haghani. For all their flaws and behavioural quirks, people might be capable of learning from their costliest mistakes. The rapid growth of index funds, in which investors settle for an average return by holding all the market’s leading stocks, suggests as much. 1.4677529 -1.1817579 0.9 NA NA NA
1_7_82 4_smp_intro_2006_2019 4 5 Non-economist While household dwelling investment continued to decline over the first half of the year, there have been signs in recent months of a prospective improvement, partly in response to reductions in interest rates. Private residential building approvals, dwelling prices and auction clearance rates have all increased. The overall demand for housing finance has been broadly stable over the course of the year and many home owners are taking advantage of lower borrowing rates to pay off their loans more quickly. 1.5854946 1.8973666 NA 0.9 NA 0.664
1_7_82 4_smp_intro_2006_2019 3 4 Economist While household dwelling investment continued to decline over the first half of the year, there have been signs in recent months of a prospective improvement, partly in response to reductions in interest rates. Private residential building approvals, dwelling prices and auction clearance rates have all increased. The overall demand for housing finance has been broadly stable over the course of the year and many home owners are taking advantage of lower borrowing rates to pay off their loans more quickly. 0.0000000 0.8017837 NA 0.9 NA 0.664

5 Session information

The session information for this program is:

sessionInfo()
## R version 4.0.3 (2020-10-10)
## Platform: x86_64-w64-mingw32/x64 (64-bit)
## Running under: Windows 10 x64 (build 17763)
## 
## Matrix products: default
## 
## locale:
## [1] LC_COLLATE=English_Australia.1252  LC_CTYPE=English_Australia.1252   
## [3] LC_MONETARY=English_Australia.1252 LC_NUMERIC=C                      
## [5] LC_TIME=English_Australia.1252    
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
## [1] kableExtra_1.3.4 knitr_1.33       dplyr_1.0.6     
## 
## loaded via a namespace (and not attached):
##  [1] highr_0.9         pillar_1.6.0      compiler_4.0.3    tools_4.0.3      
##  [5] digest_0.6.27     evaluate_0.14     lifecycle_1.0.0   tibble_3.1.1     
##  [9] viridisLite_0.4.0 pkgconfig_2.0.3   rlang_0.4.11      DBI_1.1.1        
## [13] rstudioapi_0.13   yaml_2.2.1        xfun_0.22         stringr_1.4.0    
## [17] httr_1.4.2        xml2_1.3.2        generics_0.1.0    vctrs_0.3.8      
## [21] systemfonts_1.0.1 webshot_0.5.2     tidyselect_1.1.1  svglite_2.0.0    
## [25] glue_1.4.2        R6_2.5.0          fansi_0.4.2       rmarkdown_2.8    
## [29] purrr_0.3.4       magrittr_2.0.1    scales_1.1.1      ellipsis_0.3.2   
## [33] htmltools_0.5.1.1 assertthat_0.2.1  rvest_1.0.0       colorspace_2.0-1 
## [37] utf8_1.2.1        stringi_1.5.3     munsell_0.5.0     crayon_1.4.1