Friday, August 16, 2024

Generative AI Poses Evaluation and Learning Issues as Did Calculators

Educators might say we are at a point with student use of generative artificial intelligence that resembles the teaching and learning of mathematics once calculators were available for use, which is to say a mix of positives and negatives. 


Study Name

Publication Date

Publication Venue

Key Conclusions

The Effects of Calculator Use on Mathematics in Schools and in the Certificate Examinations

2008

Educational Research Centre, Ireland

Calculator use can enhance problem-solving and conceptual understanding, but it's essential to maintain a balance with mental math and paper-pencil skills.

Calculator Use and Mathematical Thinking

1998

The Mathematics Teacher

Calculators can support higher-order thinking skills when integrated effectively into instruction.

The Impact of Calculator Use on Students' Algebraic Thinking

2005

Journal of Research in Mathematics Education

Calculator use can facilitate algebraic reasoning and problem-solving, but it's essential to focus on conceptual understanding.

Calculators in Mathematics Education

1992

NCTM Yearbook

Provides an overview of calculator research and recommendations for effective implementation.


To the extent that traditional exercises, such as writing essays, has been a proxy for testing understanding while arguably improving thinking and writing skills, generative AI poses issues, especially as learning moves away from in-person to online formats. 


In principle, generative AI could enhance creativity, but could also reduce originality; improve writing skills or leave them undeveloped; replace thinking skills or sharpen them; allow personalized learning but also possible plagiarism. 


A  key remaining question is how generative AI affects learning, namely, how humans acquire new skills as they perform tasks. One recent study, for example, involved a thousand students using two generative AI tutors, one that mimics a standard ChatGPT interface (called GPT Base) and one with prompts designed to safeguard learning (called GPT Tutor). 


“Consistent with prior work, our results show that access to GPT-4 significantly improves performance (48 percent improvement for GPT Base and 127 percent for GPT Tutor),” the researchers say. “However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17 percent reduction for GPT Base).”


In other words, the researchers say, “access to GPT-4 can harm educational outcomes.”


“Our results suggest that students attempt to use GPT-4 as a ‘crutch’ during practice problem sessions, and when successful, perform worse on their own,” the study says. 


One college professor I spoke with noted that her work had switched almost exclusively to online teaching and online testing formats, which means that GenAI use cannot be prevented. 


She noted that GenAI obviously is being used for essays and tests, which does make it harder to assess actual student comprehension of the material, to say nothing of critical thinking, reasoning or writing skills development. 


But one way or the other, schools are going to have to cope with GenAI and its impact on teaching methods, learning and assessment. 


Ultimately, that all will be reflected in labor force skills as well. But cognitive skills will continue to be vital for some industries and job roles, compared to others. 


Industry

Job Function

Higher-Order Cognitive Skills Importance

Technology

Software Engineer, Data Scientist, AI Researcher

High

Healthcare

Physician, Nurse Practitioner, Medical Researcher

High

Finance

Financial Analyst, Investment Banker, Risk Manager

High

Consulting

Management Consultant, Strategy Consultant

High

Education

Teacher, Professor, Curriculum Developer

High

Law

Lawyer, Judge, Legal Analyst

High

Marketing

Marketing Manager, Brand Strategist, Market Research Analyst

Medium-High

Sales

Sales Manager, Account Executive, Sales Engineer

Medium

Manufacturing

Operations Manager, Quality Control Engineer

Medium

Retail

Store Manager, Buyer, Visual Merchandiser

Medium

Hospitality

Hotel Manager, Event Planner

Medium

Customer Service

Customer Service Representative, Call Center Agent

Low-Medium

Administrative Support

Office Assistant, Data Entry Clerk

Low


No comments:

Agentic AI Could Change User Interface (Again)

The annual letter penned by Satya Nadella, Microsoft CEO, points out the hoped-for value of artificial intelligence agents which “can take a...