Transcript of Question & Answer Session Panel participation at the Economic Implications of the Digital Economy Conference

Hello everybody, and welcome to the dinner panel. My name is Andrew Charlton, and I'm joined by three panellists who need no introduction. Because you know them all so well, I'm going to try my best to tell you something about them that you might not know. On my left, Guy Debelle, Deputy Governor of the RBA, and also a former popular DJ in the Canberra area, who was called DJ Guy. On my right David Gruen, head of the ABS, but also G20 Sherpa who was accosted, sanctioned, for illegal swimming at a G20 event. And Danielle Wood, not only the CEO of the Grattan Institute, but also a former card counter at the Adelaide Casino. Please welcome your guests.

The format of this session is we're going to ask each of the panellists to speak for five minutes and then take questions from the floor. I might start to get the panel kicked off with an observation and a question. The observation would be that today we had a very interesting set of discussions around the digital economy and one of the themes that came very strongly out of that discussion is that we don't measure the digital economy as well as we might. But the question that I want to pose is before we get to the question of how do we measure the digital economy? And could we do it better? Can we spend some time thinking about the 'what'? What is the digital economy? How do we define it? And I think there's a lot of different definitions of the digital economy out there.

There'd be some people 10 years ago, who would've defined the digital economy in terms of a handful of companies. There might be some people who define the digital economy, according to a set of ANZSIC codes. There might be some who think about the digital economy much more broadly, a component of what every firm does. On the labour market as well, I think there are lots of different ways to think about the digital economy. You can think about tech workers as being workers who work for tech companies. You can think about tech workers as being workers in a prescribed set of ANZSCO codes, or you can think about, as Angela spoke to us about today, technology being a part of everybody's job imbued in the tasks that they do, whether they are a hairdresser or a software engineer.

I think this question of how we define the digital economy matters. It doesn't matter in the sense that we need a single answer, but I think it matters in the sense that the way you think about the digital economy has an impact on important questions, like how you measure the digital economy, how you think about the performance of the digital economy and how you think about some of the potential policy responses in the digital economy.

Just starting with that thought about measurement and performance. Depending on the way that you define the digital economy, Australia is either right at the bottom of the OECD in terms of our performance in the digital economy. If you think just about those limited set of ANZSIC industries that some people use to describe a shorthand for the digital economy, Australia's share of value add in those industries is very small, right at the bottom of the OECD. If you think about embodied technology in some of our industries like mining and agriculture – a much broader definition of technology – we are on some measures, much closer to the top. It matters, I think, as well in skills and in the labour market. Think about quite a narrow definition of the digital economy, then we have really severe skill shortages as measured by vacancy rates. If you think about a much broader definition of the digital economy, a much wider set of occupations, actually those vacancy rates are pretty much around the national average.

For my two cents worth, I think there are lots of policy questions which lend themselves to a much broader sense of what the digital economy is. Dan Andrews, who's here, and I did some work last year using data from Xero, anonymised data from the small business accounting platform Xero, looking at digital adoption during COVID. And we saw a huge response from not tech businesses, not big businesses, but from very small businesses in Australia to COVID in terms of technology adoption. The remarkable thing about this data is you can see the firm's P&L, so you can see their expenditure on ICT and we saw a 13 per cent increase in Australian firm's expenditure on ICT amongst small firms. And we also saw that those firms that invested more in ICT were more resilient through the crisis.

I think there are lots of reasons for us to think broadly about the digital economy, but to be mindful when we're thinking about measurement or thinking about policy, what do we mean when we talk about the digital economy? So first I'm going to ask Danielle to give us her thoughts on these matters.

And he says that because he doesn't know what I'm going to say, yet. And actually no one knows what I'm going to say, because when I raised what I thought I was going to say, everyone looked a bit horrified. I was going to go into all sorts of scary policy issues, but I'm actually going to keep it much more sanguine, but hopefully still raise some interesting points. I'm coming to this event as someone from outside government, but someone who's very passionate about policy and evidence-based policy. So I wanted to make some reflections on what I think are the really exciting opportunities here but what I also see as some of the challenges to realising that vision that we've been talking about today of really data-driven policy getting adopted.

First on the opportunities. I always say to my staff and my potential staff, there is no more exciting time to be an economist in the policy world than right now, and Ccertainly, if I compared it to when I started in the profession 20 years ago, the tools and the datasets that we have available to answer important questions have really grown exponentially. So I did want to recognise particularly David and the ABS and others as well, I think making available the linked administrative datasets is a game changer. I think this is potentially a transformative moment for policy. So to give you just a couple of examples, two days ago, Grattan put out some work on out-of-pocket health costs. We can see in the headline data, they've grown substantially over the past decade. We're interested in why, and who's paying them. So what we can see using the MADIP data is low income people are disproportionately hit, people with chronic conditions, and we can also see where it's coming from. It's a tale of specialists that are charging two times, three times the schedule fee. So that allows us to answer policy questions we couldn't answer before and to craft better policy solutions. I think that's incredibly exciting.

The other reason I think it's exciting is because I've been personally shocked at how few Australian academics are interested in doing work on Australian policy. Obviously we have some very notable exceptions in this room and we have them on the VCU today as well. But I think one of the barriers is, we haven't had the cool datasets to ask the kind of questions that academic economists that are really empirically driven like to ask. And so I'm really hopeful that is really going to be a shift in the paradigmand I do see that as really complementary to what I know is the excellent work going on within the public service on these datasets right now.

So those are the opportunities. I've got four potential sticking points. One, I think the data culture within public service still has some way to go. I think David has been leading the way with his enthusiasm in terms of making this data available, but we've still faced challenges in terms of getting every agency whose data you want to use to sign off in a timely way. To me really good policy analysis is nimble. Often, you're responding to a question that's very timely and so being able to turn this around and know that you can get approvals to do things quickly is really important.

Second is how receptive decision makers and the public are to this type of analysis. So I was really heartened today to hear about what people perceive to be the shift post-COVID. The recognition of the value of expertise and the recognition of the way data has helped shape the COVID response. But I think we do need to be wary that we can easily slip back into the world pre-COVID. And there was a period where there was a real scepticism of experts – the famous ‘was Michael Gove lying?’, we've had enough of experts – but there was kind of a general sense of declining interest in expertise and all the kind of data stuff that goes along with that. So I think it's incumbent on all of us to be really good at the storytelling with data to kind of bring people along with us.

Third is how do we deal with the growth in bad analysis? So as we get more data, there is also going to be a lot more questionable black box modelling that goes on. Good old days, we would just see people misusing the input-output tables, and everyone knew what was going on, but nowadays there are much more complex models. And I think it's really hard – it's hard for the public, it's hard for the politicians, it's hard, even for the public servants, with the expertise to pull apart what's happened. And I think it really matters because it can shape the public debate. So I think back to debates around negative gearing, one piece of questionable modelling, there was one piece of work that suggested a $2 billion tax change was going to wipe $20 billion off GDP. It totally, doesn't meet any sense check, yet it runs the front page of the national newspaper. It's quoted in parliament, it's tweeted by the Treasurer, it effectively derails a policy discussion.

So I think there's interesting questions about how we deal with that. I've spoken to Treasury before. I'm not sure how excited anyone is about this idea except for me, but I would love to see something like a code of conduct for modelling. A lot of state governments have it for cost benefit analysis, and I'm talking really basic stuff here. Have you been transparent about your key assumptions? Have you detailed your methodology? And if it's produced by a consulting firm, have you had the partners that are accountable for the quality, put their names to the piece of work? These are very basic checks and balances. Public service could demand it for work done for them, and it provides us some guardrails for third party pieces of modelling.

Finally, sorry, I'm probably going way over time here. How do we attract good people with these skills into policy making? I think universities are increasingly producing people with more and more data skills and that's brilliant. We spoke a bit today about how they're quite attracted to going to the tech companies and I've certainly observed that. I think companies that have exciting proprietary data sets are attractive to these people, but my observation is many of them are really enthusiastic about policy and they actually want to be doing this kind of really interesting policy work. So I think public service agencies have a huge opportunity.

First, you need to give people the tools. So whenever I've had data scientists that have left and gone to public service, if they have a complaint it's ‘I couldn't use R because it wasn't seen as secure’. That is insane. If you're hiring someone who that is their skillset, they need to have the tools. Or my computer wasn't fast enough or I couldn't do … So give people the tools. But second of all, they have to be doing the meaty policy work. So the thing that really concerns me, is that if we see increasingly work getting shunted off to the consultants – and the really interesting, important work – we're going to lose the capacity to attract those people into the public service. So I think that really needs consideration because these people will be the future. And they're going to be hopefully driving this next wave of policy making that I'm talking about.

Thank you.

Thanks Danielle. David.

Thanks Andrew. So about a year ago, we were thinking about running a conference on the economics of the digital economy, because we thought it was a topic that was worthy of more attention and Michael Smedes, who has been very influential in organising this conference said what are we going to do? And I said, well, I'm having dinner with Guy Debelle in a couple of days, so why don't I pitch to him the idea that maybe the RBA could do it jointly with us. And Guy jumped at the opportunity, so I'm very grateful to the RBA for hosting us here in their training college in Kirribilli. And we had a new idea, which was we might run a panel in the middle of dinner and we thought about who do we want on this panel. And I thought, wouldn't it be good if we could get Andrew Charlton to moderate it? So I sent Andrew an email and he said, sure. He didn't even know who was on the panel when he agreed, which was good.

And I thought Guy should definitely be on it because Guy's always got good things to say, and the fourth person (I was going to be on it, that went without saying). The fourth person who I wanted to be on the panel was Danielle. So I'm extremely grateful for the other people who I asked that they agreed to be on the panel. This is a bit of an innovation and if it goes well, when we run other conferences, we'll do something else and the question is whether I'm going to continue to have a batting average of 100 per cent, but we'll put that aside as something for the future.

Now Andrew has done many things in his life, but he was at one point a G20 Sherpa. I just don't have any stories about what he got up to, so I won't share them with you.

Now we've talked a fair bit today, and we'll talk some more tomorrow about the kind of difficulties that measurement throws up, and obviously they're pretty central for the concerns of the ABS. We talked a bit earlier today about what progress we could make in terms of measuring the gig economy and there, I think, we can definitely make significant progress and the government has funded us to look at that more carefully. I talked a bit about that today, about the fact that we are going to do this obviously in concert with our international partners, because we want it to be part of the labour force survey, and we want those questions to be internationally comparable.

So I think on finding out more about gig workers and finding out the circumstances in which they work, the extent to which they're second jobs or third jobs … I think the news is good. I think it's a matter of nailing down the questions and getting agreement. But I don't think those issues are insurmountable at all and I'm looking forward to making progress on that. Trying to get this stuff into the national accounts is obviously an order of magnitude more difficult. As Michael said, the latest national account system dates back to 2008, hence it's called SNA08, and the update presently being worked on is due to go before the UN in 2025.

So there's a big gap between one update of the national accounts and the next one, 2008 to 2025. There is going to be an attempt to at least clarify a many of the issues that we've been grappling with. So the update aims to clarify the treatment of free products, record and value free products in a satellite account, work out how to deal with digital intermediary platforms, and include data in the scope of the new SNA.

For those people who were here for Michael Smedes' paper, this is no easy task. There are conceptual difficulties about how you treat data. Is it an asset or is it something that gets used in the production process? Some things may have a long life, but clearly the data that Google maps gets from you about the traffic, about how long it's taking you to drive down a road, that's useful for about an hour, because it's going to be overridden by somebody else's data. It doesn't mean it's not useful, it's incredibly useful. And I'm extremely happy to share my data with Google maps but. these are issues that are genuinely hard, not only in terms of measurement, but also in terms of trying to fit it into the current frameworks.

Then if I can put a pitch in for Kevin Fox's paper tomorrow, attempting to come up with estimates of what people value these free products at, some of these estimates are enormous and it's interesting. The digital economy is not the only place where there are free products, but it is a source of quite a lot of them. And some of the estimates that Kevin puts forward in his paper suggests that this stuff is really valuable for people. We have to work on these measurement issues, but there are some that are clearly manageable and others that are an order of magnitude more difficult.

I want to conclude with an issue that I think has come to the fore with a huge rise in the quantity and quality of datasets generated by the private sector as a consequence of the digitalisation of their businesses. And that issue is the enormous public value that can be generated by appropriate public use of these private sector assets. Let me give an example. This is just one example, but I'm pitching something that we've just done. We have a new household consumption indicator, which we first published last month and we're going to be publishing monthly. It uses de-identified aggregated data from the major banks. At the moment, the only monthly indicator of household consumption is the retail trade survey, which covers about 30 per cent of the national accounts measure of household consumption, which is the household final consumption expenditure.

So retail trade covers 30 per cent. With this de-identified aggregated data from the banks, we can cover 68 per cent of household consumption. With time, we'll be able to raise that even further. And obviously we can reduce the burden on the people we ring up and hassle to answer the retail trade survey. We're not about to end the retail trade survey, but that's on our agenda. Once we've got this indicator up and running and we are comfortable with it and we've got a long enough run of data that it's settled down, then we will think about getting rid of the retail trade survey.

This private sector data, which has been collected for firms' legitimate commercial interest, but they're not data firms, they're banks in this case. The proposition that I want to put forward is that I think as part of their broad social responsibility to the community, big firms with data assets that they've collected for their own purposes, I think that we need to think about the extent to which they have a responsibility to share that with the public sector, with appropriate privacy and security safeguards. It's worth thinking about, to the extent that can generate public value, what are going to be the commercial arrangements between private sector firms that have big, valuable data assets and the public sector? So let me end there.

Just on that point, there's not only an externality in the use of company's data, but individual data as well. There's some social value beyond the private value, which we are at risk of not unlocking with our current framework. Guy?

Thanks. I'm not sure what territory is left after those two, but I'm going to try and head into some other territory. One thing I want to come back to is the measurement and actually, at a conceptual level, I find it interesting. Some of you may know of Geoff Harcourt, who just died last month. I actually had the privilege of coordinating a testimony to Geoff at Adelaide Uni last month. Geoff, famously, was a major protagonist in the two Cambridges debate. That was in through the '50s and '60s, and it's actually directly relevant to the questions we are talking about now. Digital technology wasn't a thing then, but it was about technology. So the two Cambridges are Cambridge England and the wrong Cambridge, which is the one I went to, which is Cambridge Massachusetts, which according to Geoff, was the losing Cambridge in the Cambridge debate.

Well, he was from the other Cambridge.

Yes, he was. He always claimed victory. A lot of it is about measurement of capital. So this is about capital, more generally of which data and digital assets is a thing. But it was really about … the Cambridge England aspect of it is that it's really about the embodied labour. And if you think about the capital is all about the embodied labour, and they thought it was always a very false distinction that you have between capital and labour, and actually this applies in this space as well, because in the end actually it's becoming – maybe this is not quite so true now – but in the end it's all generated by someone doing something.

Now with machine learning, that's maybe starting to push the boundaries of that, but the Cambridge England debate was actually everything is reducible down to the labour input into something. It's a slightly more challenging concept now with the concept of machine labour and maybe once we hit the singularity it will be a completely moot discussion. But anyway, I do actually think it's interesting having just spent a bit of time rereading some of the Cambridge controversies, that actually it's the same conceptual issue that we are confronting now.

Now I'm going to go and invoke someone from the wrong Cambridge that I went to, who's pretty famous in this space, which was Bob Solow, and just repeat a few things that Bob said. I luckily I took a class from him. So Bob, obviously one of the more famous quotes of his, which is relevant in this space, is that the computer revolution is evident everywhere except in the data. And that's actually, to a large extent, the issue we're dealing with today. So we can see all of this stuff around and it's clearly there, but we just can't see it in the data.

If you go back to Bob's original article where he has this plot about productivity growth, there are three outliers. He says: ‘Well, one plausible explanation of why these three outliers are there is because the data's wrong’. Turns out that was indeed the answer. It was probably simplifying something for which he won a Nobel Prize down to something a little too simple, but anyway, that's roughly the point.

So when we are thinking about productivity growth and technological change, and it comes to your point about where Australia ranks in some of these global comparisons, Andrew, is that we are making that assessment on data, which is not necessarily suitable.

Now I am going to invoke my DJ career. One of the shows that I used to have on 2XX in Canberra was called ‘Know Your Product’, and that's very much what we need to be aware of here. We need to know your products. We need to know the product we have now but we also need … when we are going and getting these other data sets from all these other people, we need to know what that product is as well. One thing we have to be cognisant of and a risk we have to avoid is basically saying that what's measured is real and if it's not measured, it's not real or it doesn't matter. And we are recognising, at the moment, that we have these gaps in measurement, but just because we can access a dataset for something, that makes that real, but if there's a whole bunch of other stuff that we can't easily access data, that doesn't mean that it's not real. Even including in the real GDP concept to give it a statistical point.

I suppose that brings me on to the next thing which I was going to invoke which is relevant to this, is the ability to extract signal from noise in the data. Nate Silver, who we were talking at dinner a bit earlier, has his book on signal versus noise. On 538, which is now Nate's main way of communicating with the rest of the world, they have this segment quite regularly called ‘Good Use of Polling, Bad Use of Polling’. And they put out, here's a survey, here's the question they ask, is that good use of polling, bad use of polling? And I actually think that's a question which needs to be asked of the data: Is this good use of data or bad use of data? So when someone's gone out and collected this data, is this good collection of data, bad collection of data.

One point which comes out in your paper, Mark, is the framing of the way the question is asked actually really matters. So you can get the data, but how was the question asked? How was the question framed which generated that data? That actually really matters. In the whole system of national accounts, you have had a whole long history of thinking about how those questions should be appropriately asked to generate the concepts that you actually want to measure. When you go out and talk to and access these private data collections, they haven't had that same motivation in asking those questions.

I assume they're mostly asking those questions as to what is the question I can most monetise, or the answer to the question that I can most monetise. Maybe that gives you the same answer, but maybe it doesn't. So when we're accessing all these other datasets which haven't been generated with the same diligence and rigour that the system of national accounts has been generated, then I think that's something we certainly need to be cognisant of.

The other point which was touched on in some of the comments you made David, and is evident in, at least, what I took from your paper, Michael, is this whole issue about there's (a), the non-exclusivity of some data, not all data, but non exclusivity. So if I'm a private sector company, I've generated this data, and then you, ABS, comes along and says, 'Well, I'd like to use that, thanks very much', and then actually that makes that data then readily available to all my competitors who hadn't bothered to get off their arses and collect this data. That's an interesting question to be contemplated.

Secondly, there's the whole network benefit aspect of it, which the tech companies are, I suppose the most clear embodiment of that. The network externalities really matter. If I get in front in that space, and one of the ways I might get in front is because the ABS happily uses my data that may give me a bit of an extra kicker, then I get to really benefit from those network benefits. Maybe that's the way you incentivise people to do it, but there is a sort of winner takes all aspect to it. That once you get in front with the network benefits, you tend to stay in front and move further in front.

Then there's also the issue which is the large option value of data. My sense is that we have a lot of companies collecting data and they don't actually know what the value of the data that they're collecting is. They may have collected it for one particular purpose but actually there's a whole latent option value of that data sitting there, which they're not able to realise, and actually potentially with the ABS, working with them, they may be able to actually see there's a hell of a lot more option value that's realisable from that data. So how do you deal with that from a public policy point of view. If you're basically unleashing that for them and they get all the benefits from that, is that the way you wanted things to fall? Or should, as you're working with them in refining the data, making it more usable from your side, but at the same time, making it more usable for them, do you want to be retaining some of that potentially realisable value of that data?

There's a hell of a lot of opportunity out there, with all this access to all this data that we potentially have now and the way that the public sector can capitalise on it. I completely agree with what Danielle said about evidence-based policy. Based on all this data, it has surely got to be better than policy based not on evidence. That's absolutely got to be the end objective, but as you're interacting with the private sector in accessing all the value in this data, I think there's some pretty interesting questions that come up along the way. So, they're just some of the ones which came to my mind.

Good. Terrific. Thank you. We're going to take some questions from the floor. So please have your questions ready. I might get started with a question to David. David, I'm taking you into your world as the head of statistical agencies. When you're engaging with your counterparts around the world, in what, at least in my mind are very salubrious events, when you look at practises around the world, who do you think does this well? What do you think we can learn from? Are there examples out there of other countries who deal with some of the issues that Guy was talking about, get the incentives right, get the structures right, are delivering things that should be markers for us to look forward to? What's the best practise out there?

That's a good question. I think one of the differences between countries is probably the arrival of the pandemic meant that a lot of statistical agencies started looking at the possibility of alternative sources of data and plenty of them have gone for mobility data, telecom data.

So there's a wide range of practises and I'm not confident I know what best practise is because this is moving so quickly and people are, I think, learning from each other. I think in some ways we are doing things that some other countries are not doing, but then there are other things that others are doing. And it's partly the legal environment that statistical agencies find themselves in, what can they do. What I'm impressed by is the fact that there's been a lot of commonality in terms of a sprint to try and find new data sources and provide closer to real time information about what's going on. And we are certainly part of that, but I wouldn't want to pick any particular country as being in the Vanguard. Although, the minister suggested this morning that it was Estonia though. He then said, 'but not necessarily'. So, he wasn't going to commit to that.

Certainly the work that the ABS did in this crisis as has been said today, was a huge step on from the data available to the government in the last crisis. I remember, I worked in the Prime Minister's office during the last crisis and one of my counterparts and very good friends is a fellow called Lachlan Harris, who you will remember at the time as the Prime Minister's senior press secretary and Lachlan's family owns Harris farm. So our version of real time data during the GFC, was Lachie calling up his dad and saying 'how are the tools running today?' So he was doing a slightly more sophisticated version of that really. I think we had a question over here.

Hi. David started off the conference today by giving a story about the world's largest companies 25 years ago. So 25 years ago, I was in the British Civil Service. I was working two days a week at home. I was using the internet and email to do my work with my colleagues in London. So, I'm kind of wondering in one sense, how much has changed, but I'm kind of also wondering, we were talking at that stage maybe about the information economy rather than digital economy. Have we advanced our understanding over the last 25 years or are we, to quote another Saint's song, given Guy did one, I'm stranded. Have we advanced our understanding over the last 25 years? Or are we still just spinning wheels?

Who's going to answer that one?

I think that's one for the statistics boffins.

I reckon we've got better data and so it kind of depends. I mean, what about the stuff that Raj Chetty's doing? What about that? So, are getting big datasets and applying them to micro problems and working out who it is in …

So I think it's messier, because plenty of these datasets, as I think Guy said, have not been designed with a specific purpose that they're now being dragooned into. So they've being designed for some other purpose. But I think the granularity and the fact that it's much closer to real time, to make another example, the policy makers in the COVID crisis had a much better handle on what was going on in the labour market than the policy makers of whom Andrew was one and I was another, in a different entity for the global financial crisis.

So I think we have got a much better information base. I'll let Danielle answer, but I think the promise of administrative data is that you'll be hopefully able to get a much better handle on microeconomic problems than we've had in the past.

Absolutely. I think when people started to get excited about data, often the questions were driven by what datasets were available. So they were like, 'I've got this dataset on ride sharing, so I'm going to ask this particular question'. I think the exciting thing about administrative datasets is you can actually start with the policy question you want to ask and now the data is sitting there to allow you to answer that very specific question. So I think that is a real game changer.

I think you might have been ahead of your time 25 years ago. Certainly, the data suggests not many people were working from home on the internet at that point in time. Now, a lot are and as I said, the tools and the information that we have at our fingertips is dramatically different.

There's measurement without theory andI think a lot of the data before now has been measurement without theory. So the advantage of getting the ABS involved in it is actually, you're providing the theory not ahead of the measurement, the measurement's still there, but actually providing the theory to guide the measurement.

And I completely agree with Danielle. A lot of my peers in grad school, the way you did your thesis, was you found some funky dataset, which no one had basically, and that was how you did your thesis. It didn't really matter what question you asked, as long as you found a dataset which someone else hadn't found before, and that would do it. Whereas now, it's actually, no, I actually want to answer this question and so what data do I bring to bear on that question? That's quite a different way of doing it. So the constraint before was the existence of the data. Now the constraint, which is the appropriate constraint, is what is the question you want to answer.

I used data from ‘Who Wants to be a Millionaire’, the game show, for my thesis, which shows you how desperate I was to find a cool dataset.

Dan.

Thanks. So Danielle, you picked up that we've kind of had a good crisis, right, in terms of data. I think that's true. I would say though, that there was a bit of a forum for that. There's a quiet revolution that happened in Australia. So David, obviously you being appointed to the ABS was a game changer. But I think back to the days of DIPA, the data integration partnership for Australia, which Mark mentioned, the BLADE data set was set up. The reality of DIPA is, it reallocated a lot of money to Treasury to actually set up a gun applied micro data team, which could then be spun behind real time data analysis during the crisis.

So in a sense, I think it's true that we've done well, but there was a movement before and that fight was often bitter. And everyone who talks about evidence-based policy in the public service, only a couple of them show up when it matters. So my question is, in say 10 years' time, when we have the next economic crisis or whenever it is. Guy, you may be Governor …

It might be tomorrow, the next economic crisis.

How do we actually keep going from here? What do we need? What would be the one piece of data that each of you'd want to actually improve decision making? Because what I worry about is that, things calm down, David retires, does whatever he wants. Maybe joins E61. [Laughter]… just wanes, we go back to the old habits and then boom, all of a sudden, financial crisis hits. All of a sudden, we don't have firms and banks linked together and that takes a long time to do, so how do we keep this going? Let's get real here for a second. David, I've got a split personality when it comes to data integration in Australia, you know this, and I'm giving the ABS credit now, but there is a history here where not a lot happened over a long period of time. So, how do we actually keep this going, so that the next time we get a big shock, we can have policy that's guided by evidence like we've had during this crisis?

Good question.

So it's not obvious to me Dan, that you're basically asking more about what I would loosely describe as macro data. It's not obvious to me that's actually where the ABS should be committing resources. So I think – you do have a budget constraint David …

We do. [Laughter]

But the serious point I want to make is we've got to think … most of us are probably economists, so we've got to think about it as economists. So we have scarce resources to allocate to this problem. So what is the data that we feel is going to be most consequential? Not obvious to me that the answer to that is better macro data. It might be, but it's not obvious to me that it's actually that. I could argue that actually better health data and giving people access to that may be a more appropriate use of public sector funds. So, I'm not going to answer your question, Dan, because I think the answer isn't at all obvious, but it's actually a pretty important question and it partly comes to the point I was saying earlier which is just because the data exists, it doesn't mean that that's where I should be throwing the money. You've got to think, where is it I feel that could be most consequential?

So you mentioned BLADE or longitudinal studies more generally. One thing about longitudinal studies, you've actually got to start them at some point, and so, there's no point saying, well, that would be nice if we had a long time series or long panel series on this. If you haven't thought about it before, you're only going to start collecting that now. But that may be a very, very worthwhile investment to actually start doing that today so that having thought about what sort of questions are they going to be potentially able to answer with that panel data set in some number of, maybe years, decades' time. So I do think I would frame your question slightly differently. I think it's a question of thinking about where do you think the most value add is to be obtained, and it's not just because I want to deal with the next financial crisis. Maybe I want to deal with the next health crisis.

Who wants to intervene.

I'm happy to answer.

I was just going to offer some counterpoint to this obsession with real time data.

Yeah, that's where I was going.

The insight during the GFC was a theoretical one that people already knew, which was, we need to pump the money out to people who are going to spend it, and so it goes around. That wasn't something that you truly needed real time data to do. Yes, you might want to track it and see how well it's going, but that was the thing. And during COVID, it was the understanding of epidemiology and things like that, that really should have been driving it rather than making it up as you're going along.

Because certainly, at least in the economic sphere, there's no reference for this. But if you had the long-term development of models, which were based on data, but it was historical data and having all of the access to that that allowed people to understand how do diseases spread and what are things that we need to do that are going to restrict that, and fundamentally, trusting it, because the lags, even in the real time data, were so long. So I guess what I'm saying here is, it's not real time data. It's the capital you build up, analysing the long, the last 10 years of data to understand what fundamentally drives the economy. And so, it's the theory that you need to have developed and then you can use the real time data to track it. But actually, if your theory's good, that's 90 per cent of the battle.

I'm a bit sceptical about that. I think the real time data was actually incredibly important here. You can run all the epidemiological models in the world, but we needed to understand how it was playing out on the ground. So I think the bank transaction data is incredibly important. That would be my answer to your question. I think it is actually seeing spending, and AlphaBeta was putting out work, which we were grasping on because it was incredibly important to see how it was impacting different sectors. The Xero data was important to understand what it was meaning for jobs, the payroll data.

So I actually think crafting the policy response does require that understanding of how it's actually playing out on the ground and I'm not sure your theoretical model can get you there. But just very quickly I think it's really important that we have access to that data and the capability to use it, and it has to be sitting there, but in preparedness, I would also like more thinking done before the next crisis in terms of actually tailoring the policy response.

So my criticism this time around was, more that, particularly for central specific supports, we just kind of went back to rolling out construction spending. So we can talk about payments or vouchers but on the business side, construction is not the best stimulus. And I think we need to be much more creative about what we do in terms of designing programs and you can't do that on the spot. You actually have to do the work in advance. So I think there is a lot of preparation we could do, and yes, we'll have to tailor it at the time, according to the nature of the crisis. But there are things we can think about now to be ready for the next one.

Can I back that up? I think governments were very data hungry in thinking through their policy response. We were talking to governments who were looking for very high granularity regional data to think about different programs. I totally agree with your point that those programs probably didn't have the preparation that would make them even more effective or to think about what the best policy response was. But I did see a real hunger to use data to inform that process, albeit it didn't always get the right outcome.

You have a question over here as well.

Thanks very much and thanks for the panel's observations. I have a question for Danielle, but before I ask it, I just want to say that I really reflect on the 90s, we talked about evidence-based policy making and then the language shifted slightly to evidence-informed policy making. And I suppose what I'm saying is that, whatever evidence or data we have, it also depends on the politics of the situation about the policy that's actually implemented.

My question, Danielle, is to you completely different coming from the tertiary sector and trying to stand up for the universities a little bit, I really liked your comment that we want to train and educate our students to be the next generation of people who can make good policy decisions. So it's really a question of the whole panel because Andrew, I've seen you're in Accenture now, but the big accounting firms, the big consulting firms are taking our students into research. That's where they're going now. How do we encourage them to think about good public service jobs and what training should we do? And I think having access to administrative data is fantastic because I think of the field I work in, now that we can have that, we can start to do some really good research. But what should we be doing in the universities to really assist Australia in developing good economists, good policy people, good civil servants?

That's a fantastic question and a huge question. So I think there's been a really important debate within the economic profession about how we actually teach economics at universities and rather than starting with the models and building up to relaxing the assumptions and asking the interesting questions in year three and year four, flipping it on its head and starting with the questions that people go into economics to find out about like why is there inequality or why did the GFC happen? Starting with the meaty policy debates and there's the core curriculum and others that are actually trying to do that and expose people to that in the early years. And I think actually once you give people the taste of that, they're going to want more, but then I think it's linking it in – and, and maybe it is through the administrative datasets – linking people in to the relevant agencies where they're asking those questions. I think internship programs are fantastic. Certainly at Grattan, we find, we get amazing really policy-enthusiastic students coming through. So I think the more agencies do those programs, the more you can expose people and the accounting firms are good at this, right? They're marketing to students, they're offering the internship. That's how they're getting them in. They're creating those pathways. So government agencies need to be competing on the same terms.

We've been doing that for a little while and so I completely agree with that. But one thing, to your particular question, and I'm going to pick up with what you said, Danielle. So this also came up when I was back in Adelaide talking about the way that the economics was taught when I went through with Mark. So we had a course in our first year called ‘Economic Institutions and Policy’,

which I think there were six different parts across the year and just whatever public policy issue any of the faculty wanted to talk about for six, half – this shows mine and Mark's age, it was back in the days of three term years – but six different modules effectively on a public policy issue. So this was a first year subject. So the first week of this year hadn't had any other economics except maybe one other lecture. You're given that straight up, and so it's doable. The slight challenge for possibly the university faculties actually requires a bit of work to put together a coherent course on that. The other one was, which I think is interesting in some of the latter years, in our second year statistics or econometrics class, I remember we were probably used as unpaid labour basically, but the professor who was teaching the course got us to go out and do our own. So we had to go out to supermarket car parks and we had to sit there and watch people drive past and see if they had seat belts on in the back seat.

So this was in the days at least in South Australia where that wasn't compulsory and particularly whether they had their kids in a seatbelt in the backseat, or they were just sitting on their lap in the driver's seat without a seatbelt on. Anyway …

So what's the point of that story? [Laughter]

So the point was you had to write that up as the project in your studies. So you are answering a policy relevant question. This is whether seatbelts should be compulsory. And this was actually being commissioned by the South Australian Roads Authority. I'm pretty sure. So it was a policy based question. You had to go and generate your own data in this case, and then work out what the problems were about the data and you're answering a policy-based question with data.

So at least if you had that inclination that that was something you were interested in, you're exposed to it quite early on as you are learning all the material. So, I think it's actually the exposure. You probably can do that a little more sophisticated these days, but getting your hands down and dirty with actually collecting the data is no bad thing either. So I do think that, you know, to Danielle's point, I actually think that's a pretty important part of it.

And it's good to know that. I mean, we're constrained by ethics and student safety …

Yeah. It wasn't a problem back then.

… but I think it's a great idea. Thank you.

We give them the R code now.

Yeah of course.

Jason?

So my question is I worry about real time data, because some has the negative effect of turning politicians into junkies a little bit. How do you recalibrate them into thinking about the longer term issues where big data does have some potential benefits such as climate change, productivity intergenerational disadvantage. How do you refocus politicians onto those issues?

It's not for me …

You worked in Prime Minister's office, although he was focused on all issues though, and if he had access to the data …

I would've slept even less. I think it's a really good question. I think there are lots of politicians who, as you say, are getting more and more interested in the data and overall, I think that's a really good thing. I think it's sort of a similar question to the question that Danielle posed, which is how do you make sure that this explosion of data doesn't create an explosion of spurious results and an explosion of distractions. I think that goes to some of the thoughts around quality, increasing sophistication of the community that digests and engages with this data. I thought some of Danielle's suggestions were really terrific in terms of ways to lift the standard and probably some of those are applicable as well. But I think overall it's a good thing that politicians have more data available to them. That there's a sense of accountability out there that this data will not just be used in policy formulation, but also potentially be used in policy evaluation and that has strong prospects for the future.

My question was about the use of private data.

So when we think about using say bank data or supermarket data, do we worry enough about the fact that consumers are responding to incentives as they're creating that data? So for instance, if I'm about to apply for a mortgage loan, I may think very carefully about what I'm spending my money on in my bank account. I'm going to want to look like a very responsible consumer and my patterns will look different than they might if I wasn't about to apply for a mortgage loan. Or grocery loyalty data, which is awesome. But as soon as the grocery store starts giving people discounts based on patterns in their shopping, that behaviour is going to change. Some people are going to think strategically about what they purchase. Are we worrying enough about if we're using BLADE data, people think about what they report on their taxes, they design their investments, to minimise their tax burden. Are we thinking hard enough about, how incentives are clouding the data that we're using?

I think it depends what the question you're trying to answer is. So I'm not worried about the banking data in the sense that I'm not only using the data of people that are applying for mortgages, I've got a huge dataset of everyone's transactions. So I think that's kind of a wash.

The loyalty card data is interesting and again, if I was trying to understand the impact of discounting, elasticity of demand for certain products that's actually super useful to me, right? That's the kind of variation I want. If I was using it to try and understand consumption patterns in the broad, then I think I would need to be worried about the way in which individuals are getting different signals. So I think it's always, with any data set, a matter of understanding the nature of the underlying data set, what you're collecting and then how that maps to the question that you want to answer. So I think sometimes it's an issue, sometimes it's not such an issue.

I think we should not pretend that the data that we were using in place with this, that the retail survey was a perfect dataset that was free from reporting issues. I think generally this data is better quality because individual transaction data, you're actually picking up something real as opposed to surveys, which have all sorts of validity issues.

Surely those incentives for misreporting are higher in a survey-based environment than they are admin environment?

Getting off the phone quicker, that could be something. [Laughter]

Terrific. Okay. Please join me in thanking our panellists for a very interesting discussion.